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ABSTRACT 
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Foreword 



This is a report on the New Chance Observational Study - an in-depth examination of 
parenting behavior and its relationship to children’s development in a subset of the families 
participating in the New Chance Demonstration. 

New Chance was a national research and demonstration project that provided 
comprehensive education, training, and other services intended to improve the prospects and 
well-being of low-income mothers and their children. The program’s eligibility criteria were 
designed to assure that the research sample represented families central to the welfare reform 
debates of the past 10 years: families headed by young mothers who had their first child as 
teenagers, were high school dropouts, and were receiving Aid to Families with Dependent 
Children. (AFDC was the main cash welfare program until the 1996 federal welfare legislation 
replaced it with Temporary Assistance for Needy Families, or TANF.) 

One of New Chance’s distinguishing features was its explicit two-generational focus on 
both mothers and children. Many of its services were meant to help the mothers prepare for, get, 
and hold onto jobs so that they could become economically self-sufficient and leave welfare. In 
designing the demonstration, MDRC believed that such changes, if they occurred, would 
potentially improve developmental outcomes for children. But the aim was to shape the 
children’s development more directly. Therefore, the demonstration included parenting 
education, access to pediatric health services, and an attempt to develop and encourage the use of 
good-quality child care. 

The original research plan for evaluating the New Chance Program recognized the 
importance of assessing its two-generational nature. Thus, it included several measures of 
parenting behavior and participation by the mother in the child-related components of New 
Chance (e.g., parenting education), along with various measures of the children’s development. 
Sources for the latter included surveys of parents, reports from teachers (for children in the 
research sample who were in formal child care or school settings), and a general measure of 

school readiness. 

While the initial design of the New Chance Program and its research plan went beyond 
the practices typical of large-scale field studies of this type, they created the opportunity to push 
even further. Thus, MDRC formed an interdisciplinary “observational studies team” that 
designed and implemented the New Chance Observational Study and a companion study 
embedded within the evaluation of the Job Opportunities and Basic Skills Training (JOBS) 
Program (the welfare program authorized under the legislation that preceded the legislation 

establishing TANF). 

Each of the observational studies makes a new and unique contribution regarding how 
best to measure parenting and child outcomes in a survey context. This is a question important 
to anyone trying to understand the effects on children of welfare reforms or similar initiatives. 
Much of the existing information about how children develop comes from the university-based 
laboratories of developmental psychologists, yet social policies and programs play out in the 
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everyday lives of parents and children. The observational studies transported university-based 
techniques into the field, gathering data via videotape and audiotape in the participants’ homes. 
In large part, this translation of methods and materials succeeded, and it yielded rich information 
about the strengths and limits of using regular survey interviewers to go well beyond their typical 
interviewing tasks. The information on all this methodological work is found in the second part 
of this monograph. 

The incorporation of these new techniques and measures into the New Chance 
Demonstration also allowed us to deepen our answers to certain questions. For example, using 
the originally planned survey measures, MDRC staff have examined questions about the effects 
of New Chance on parenting, and the relationships between those effects and child outcomes. 
But these analyses acknowledged the limits of data on parenting or child development that come 
from parental self-report and observations by survey interviewers. By including the measures 
derived from videotapes and audiotapes of the observational study sessions, the observational 
studies team was able to more fully explore the effects of New Chance on parenting, the 
relationships between program participation and these effects, and the role of the parenting 
effects on child outcomes. 

Interestingly, the New Chance program did have positive effects on parenting. These 
appeared on both survey interviews and observational measures. However, other influences such 
as maternal psychological well-being, including stress and depression, combined with the effects 
of parenting behavior on children. Thus, the positive parenting differences were not sufficient to 
bring about effects on child outcomes, no matter how the parenting and child outcomes were 
measured. 

While the observational study data were consistent with the survey data and thus did not 
change our judgments about the effects of the New Chance Program, they were promising in 
other ways. The observational data appear to have tapped a wider range of parent and child 
behaviors than is possible in the survey context. For example, the survey measures did not detect 
program impacts in the area of cognitive stimulation, while the more sensitive and qualitative 
observational measures did. The observational measures also added to our ability to understand 
why some children are doing better than others. Such questions and analyses are the subject of 
the first part of the monograph. 

Interdisciplinary and pathbreaking, the observational study represented a collaboration 
among a diverse team. The editors of the monograph, Martha Zaslow and Carolyn Eldred, have 
generously described the roles of all team members except themselves in the Acknowledgments. 
Each of the editors has given an extraordinary amount of time to this effort over the last six 
years, and their contributions cannot go unnoted. 

Martha Zaslow has been involved in all aspects of this work. Together with colleagues at 
Child Trends, she analyzed the interview-based measures of parenting, examined the 
relationships across different parenting measures, and conducted analyses regarding the role of 
parenting and other factors in predicting child outcomes. She also wrote or contributed to several 
chapters in the first part of the monograph and reviewed all of the manuscript several times. 
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Carolyn Eldred began the effort as the project director for the New Chance surveys at the 
Institute for Survey Research at Temple University. In that role, she worked with MDRC to 
hone the overall study design, thought through the myriad issues involved in conducting and 
recording the observational sessions, adapted the laboratory protocols for survey administration, 
trained the field staff, and directed the data collection effort. Subsequently, as a consultant to 
MDRC, she undertook the methodological reflection and analysis in the second part of the report, 
wrote the chapters on methodological issues, coauthored the chapter on study design in the first 
part of the report, and reviewed the entire monograph several times. Both editors have been 
“first among equals” on this team. 

The study owes a great debt to the energy and good efforts of the mothers who invited us 
into their homes, diligently worked through the tasks with their children, and shared their 
thoughts in the interviews. Above all else, the research team hopes that this work is up to the 
trust shown by these families. They have been open and generous; this monograph is dedicated 
to improving their lives and the lives of others in similar circumstances. 

Finally, we are grateful to the funders of the observational study: the Foundation for 
Child Development, the William T. Grant Foundation, the Smith Richardson Foundation, and the 
National Institute of Child Health and Human Development. Their commitment made the project 
possible. 



Robert C. Granger 
Project Director 
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Executive Summary 

Martha J. Zaslow and Carolyn A. Eldred 



Part I: The New Chance Observational Study 



I. The New Chance Demonstration 

The New Chance Observational Study — the subject of this monograph — is an in-depth 
examination of parenting behavior in 290 of the 2,322 families studied in the New Chance Dem- 
onstration, a national research and demonstration program operated between 1989 and 1992 at 16 
locations in 1 0 states. The demonstration tested a program model intended to improve the eco- 
nomic prospects and overall well-being of low-income young mothers (aged 1 6 to 22) and their 
children through a comprehensive and intensive set of services. It was developed by the Man- 
power Demonstration Research Corporation (MDRC) and supported by a broad consortium of 
public and private funders. 

New Chance was directed at families central to the welfare reform debates that culmi- 
nated in the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 — fami- 
lies headed by young mothers who gave birth during their teenage years and were receiving Aid 
to Families with Dependent Children (AFDC, the main cash welfare program). 1 More specifi- 
cally, New Chance focused on those who were especially disadvantaged because they were high 
school dropouts; as a group, they and their children are at high risk of long-term welfare receipt 
and economic hardship. 

The New Chance Program sought to help the young mothers (who, for the most part, vol- 
unteered for the program) to acquire educational and vocational credentials and skills so that they 
could find and keep jobs offering opportunities for advancement and reduce, and eventually 
eliminate, their use of welfare. It also sought to motivate and assist participants to postpone ad- 
ditional childbearing and to become better parents. Because New Chance focused on young chil- 
dren as well as their mothers, it sought to further the cognitive, social, and emotional develop- 
ment as well as the health of participants’ children. Child care was provided at no cost to the 
parents, on site in most places, and the program facilitated access to health services for both 
mothers and children. The program was intended to be intensive (four to five days a week for up 
to 18 months), though in practice attendance was of much shorter term and often irregular. 2 



'The law, which replaced the entitlement to AFDC with a block grant to states, requires unmarried minor par- 
ents caring for a child to live with an adult, or in a setting supervised by an adult, and to participate in education or 
training activities in order to receive welfare assistance. The law also calls for the U.S. Department of Health and 
Human Services to develop and implement a strategy to prevent nonmarital births to teens and to assure that at least 
25 percent of communities have teen pregnancy prevention programs. 

2 The New Chance Program and its implementation are described in detail in Janet C. Quint, Barbara Fink, and 
Sharon Rowser, New Chance: Implementing a Comprehensive Program for Disadvantaged Young Mothers and 

(continued) 
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Ev enting Behavior and the Two-Generational Character of New Qhfflce 



New Chance was one of only a few interventions for families in poverty that took a two- 
generational approach, seeking to improve the outcomes for both the young mothers, who faced 
multiple difficult life circumstances, and their children. For this reason, parenting behavior took 
on particular importance. If the New Chance Program improved the quality of the mother-child 
relationship, it would signify improved functioning in an important area of the mothers’ lives. It 
would also have the potential of diminishing risk and improving developmental outcomes for the 
children. Research shows that the children of young single mothers in poverty begin to show 
higher levels of behavior problems as early as the preschool years. Later in development, they 
show difficulties in school progress and achievement. 

New Chance sought to enhance parenting behavior most directly through its parenting 
classes, which provided information on children’s developmental stages, activities and materials 
to enhance children’s cognitive development, and developmentally appropriate strategies for 
shaping child behavior. Parenting classes, which were scheduled for about two hours a week, 
balanced open discussion of issues of concern to the mothers and more formal presentation of 
specific information. 

Other aspects of New Chance also held the potential for enhancing parenting behavior. 
Life skills training (like parenting classes, scheduled for about two hours a week) focused on im- 
proving the mothers’ skills in communicating with significant people in their lives, including 
children. Adult basic education (including preparation for the General Educational Development, 
or GED, test) and job skills training classes provided stimulation to the mothers, which in turn 
could be reflected in how they talked with, read to, and played with their children. Group and 
individual counseling addressed problems emerging in the mothers’ lives, including problems 
with children. Family planning classes stressed the importance of providing enough time and at- 
tention to each child in the family. The child care that the children participated in provided a 
context for the young mothers to observe care providers engaging in stimulating and supportive 
interactions with children — behavior they might then imitate. Participation in the program as a 

whole could increase the mothers’ sense of social support, which in turn could enhance parenting 
behavior. 



IIL The New Chance Observational Study; A Study Embedded in the New fhanrg 

Evaluation 

A. Evaluating the New Chance Program 

Central to the New Chance Demonstration was a rigorous evaluation of the program’s ef- 
fectiveness. For this purpose, 2,322 young women who applied to New Chance were randomly 
assigned to either an experimental group (who were allowed to enroll in New Chance) or a con- 
trol group (who did not have access to services provided through New Chance, but many of 
whom found some alternative services in their communities). To determine the effectiveness of 



Their Children (New York: MDRC, 1991). 



the New Chance "package" of services, compared with these alternative services, differences in 
outcomes between the two groups of women and their children (often referred to in this kind of 
research as the impacts of the program) were examined through structured survey interviews ad- 
ministered approximately 1 8 and 42 months after each young woman entered the research sam- 
ple. The findings from the demonstration have been presented in a series of MDRC reports con- 
cluding with the final report, released in 1997. 3 

B. The Purposes of the Observational Study 

Because of the importance of parenting behavior in the context of a program focusing on 
outcomes for two generations, a special study was undertaken, using videotaped observations of 
mother-child interaction among a subset of 290 families in the New Chance Demonstration. The 
work of a collaborative and interdisciplinary research team (described in the Foreword to this 
monograph), the observational study was funded by grants from the Foundation for Child Devel- 
opment, the William T. Grant Foundation, and the Smith Richardson Foundation, with specific 
analyses focusing on methodological issues funded by the National Institute of Child Health and 
Human Development. 

The observational data were collected during a special session following the 1 8-month 
follow-up survey for the full evaluation, on average 21 months after each member of the obser- 
vational study sample had enrolled in the demonstration. The videotapes of mother-child interac- 
tion were then coded under rigorous conditions in university laboratories. The purposes of the 
study were to (1) describe parenting behavior in this sample, especially the affective quality of 
mother-child interaction and the aspects of interaction that are related to the emergence of liter- 
acy in children; (2) examine in greater depth the program’s impacts on parenting behavior; (3) 
explore the role of parenting behavior in shaping the outcomes for children; and (4) assess the 
added value of using measures of parenting based on direct observation in addition to the 
evaluation’s survey interviews. 

More specifically the study asked: 

• What background characteristics of the families are most closely related to 
parenting behavior for the families in the sample? 

• Based on sensitive and detailed measures of parenting used in the observa- 
tional study, did the New Chance Program have positive impacts on parenting 
behavior and, if so, which program components contributed? 

• What role did parenting behavior play in shaping the development of the chil- 
dren in the sample? What was the role of such other important influences as 
the mothers’ psychological well-being and the family’s economic resources? 

The study also asked how best to measure parenting behavior in a program evaluation 
that focuses on two generations. More precisely: 



3 Janet C. Quint, Johannes M. Bos, and Denise F. Polit, New Chance: Final Report on a Comprehensive Pro- 
gram for Young Mothers in Poverty and Their Children (New York: MDRC, 1997). 



o Do measures of mother-child interaction based on direct observation substan- 
tially improve the quality of information about parenting beyond what is 
available through measures collected in the context of survey interviews? Part 
II of this Executive Summary (and of the monograph) discusses the methodo- 
logical issues in more detail. 

C. The Samp le 

The sample for the New Chance Observational Study was chosen from the seven New 
Chance sites that had the largest number of families who met the study’s criteria and could po- 
tentially participate. Specifically, the study chose families with a "focal child" between 30 and 60 
months old, an age range considered appropriate for the study’s procedures. (The "focal child" 
was the child in each family who was the focus of the interviews and assessments in the full 
evaluation’s impact research.) In addition, the observational study was limited to African Ameri- 
can and white families, rather than also including Hispanic families, because the researchers did 
not consider it possible to give full consideration to variation in parenting behavior that might be 
associated with cultural background. To study how parenting behavior related to other important 
aspects of the families’ lives, the sample was also limited to families who had completed the 
evaluation’s 18-month follow-up interview. Finally, the study was restricted to families for 
whom the observational study session could be scheduled within a similar time frame relative to 
random assignment — no more than four months after each mother’s 18-month follow-up inter- 
view. The eligibility criteria for this study were more restrictive than the criteria for the overall 
New Chance Evaluation, and the sites used in the observational study were not selected ran- 
domly from all the New Chance sites. Therefore, the findings from the observation study should 
not be seen as generalizing to the full New Chance sample. The larger evaluation sample, for ex- 
ample, included families with children in a broader age range, families from additional sites, and 
families of Hispanic origin. 

The sites from which the families were chosen were the Bronx, Detroit, Harlem, Lexing- 
ton (Kentucky), Philadelphia, Pittsburgh, and Portland (Oregon). Within each of these sites, 
families who met the study’s eligibility criteria were contacted in the order in which they had en- 
rolled in the demonstration. 

The goal was to conduct observational sessions with about 300 families — a sample size 
large enough to detect program impacts on the observational measures and to permit an exami- 
nation of how parenting behavior was related to the mothers’ education, age at first birth, years 
receiving public assistance, and other important characteristics, as well as child outcomes. Of the 
New Chance sample members who met the criteria for inclusion in the observational study, 79 
percent participated, yielding usable videotapes from 290 families. Approximately 84 percent 
were African American and 16 percent were white. Among the focal children, 148 were boys and 
142 were girls. One hundred and eighty-four of the mothers were members of the demonstra- 
tion’s experimental group and 106 were from the control group. The families in the two groups 
did not differ significantly in their baseline characteristics, so group differences found at the time 
of the observational study can be attributed to the experiences of the families subsequent to ran- 
dom assignment. Nor was there evidence of systematic differences between the families identi- 
fied as eligible to participate in the observational study who did and did not participate. 
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D. An Overview of the Study’s Procedures 



The visit to each family participating in the observational study (lasting, on average, 
about an hour) was conducted by a two-member team — a survey interviewer, who briefly inter- 
viewed the mother and guided her through a series of interactive tasks with her child, and a 
videographer, who taped the mother and focal child as they carried out the interactive tasks. The 
interviewers, who were accustomed to conducting traditional survey interviews and were already 
working on the New Chance 18 -month survey, received specialized training to follow a struc- 
tured script and to explain and administer the tasks to the mothers with minimal interference in 
the mother-child interaction. 

The observational session, or extra visit carried out for the New Chance Observational 
Study, started with the interviewer explaining the procedures to the mother and obtaining her in- 
formed consent. The interviewer then administered the first half of a brief interview with the 
mother, in which she was asked to describe, in half-hour intervals, the activities that she and her 
child had engaged in during the previous weekday. Beginning the session in this way helped the 
interviewer to establish rapport with the mother, gave the videographer time to set up the equip- 
ment, and provided further information about the mother-child relationship. 

Following a script, the interviewer then described each of the interactive tasks to the 
mother and confirmed that she understood how to carry out each task. After these initial instruc- 
tions, the child was invited to join the mother and interviewer, and the mother was asked to guide 
her child through each of the interactive tasks. The script called for the interviewer to remind the 
mother briefly about each task as she presented the props for that task. Interviewers were in- 
structed to then let the mother interact with her child as she chose, without interference. 

The tasks, devised by university-based researchers of children’s development, had been 
used in previous studies of mother-child interaction, including studies of low-income families. 
The six tasks were: 

® Book reading. The mother was asked to read and discuss a children’s book. 

The Very Hungry Caterpillar (by Eric Carle), with her child the way she 
would usually do so; 

® Blocks. The mother was asked to try to get her child to match the shape of a 
larger block by using combinations of smaller blocks; 

® Wheels. The mother tried to get her child to name as many objects with 
wheels as he or she could within the time allotted; 

© Sorting. The mother asked her child to place plastic chips of different shapes 
and colors in the empty squares on a board according to the shapes of chips 
glued onto the board in a row at the top, and the colors of chips glued on in a 
column at the left; 

® Etch-a-Sketch. The mother tried to get her child to use the knobs on an Etch- 
a-Sketch board to draw a line tracing a maze that had been drawn on the 
screen; 
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g Gift. The mother was presented with a wrapped gift — a kaleidoscope — to 
give to the child, and the mother and child then spent a few minutes opening 
and playing with the gift. 

Each of these tasks had either been used by members of the research team in previous 
studies or was a modification of a task used before. The tasks were chosen because they yield 
forms of mother-child interaction that predict social behavior and academic achievement in 
school. 



The observational session concluded with the interviewer administering the final portion 
of the brief interview to the mother, which included questions about use of child care for the fo- 
cal child; the mother’s participation in educational, training, and employment activities; and the 
family’s residential situation. The interviewer also completed ratings about the home environ- 
ment and about the observational session (for example, whether there had been others in the 
home during the session and whether the session had been interrupted), and the mother com- 
pleted a brief “self-administered questionnaire” with items concerning her subjective sense of 
well-being, perceptions of the focal child, and reactions to the mother-child interaction tasks. 

The initial instructions to the mother as well as the series of mother-child interaction 
tasks (but not the interview segments of the observational session) were videotaped. The video- 
tapes of mother-child interaction were then coded in two independent research laboratories. The 
affective quality of mother-child interaction was rated by a team of researchers at the University 
of Minnesota. Ratings of mother-child interactions related to the emergence of literacy were 
made by a team of researchers at Harvard University, based on transcriptions of verbal interac- 
tions during the book reading and wheels tasks and on the interactive behavior from the video- 
tapes for these tasks. 

E. Parent-Child Data from the Full New Chance Evaluation 

The families in the New Chance Observational Study also participated in the data collec- 
tion that was part of the full New Chance Demonstration. Thus, there is information on the fami- 
lies prior to their being randomly assigned to the evaluation’s experimental or control groups 
(“baseline data”), before any program effects could have occurred. There are also data from the 
18- and 42-month follow-up surveys, both of which included measures of the mothers’ psycho- 
logical well-being; educational attainment, employment, earnings, and welfare receipt; residen- 
tial situation; use of child care for the focal child; and fertility. 

The surveys from the full evaluation also included interview-based measures of parent- 
ing. The analyses reported on in this monograph focus on parenting measures from the 18-month 
follow-up survey, because these were collected close in time to the observational measures. The 
18-month follow-up included three parenting scales based on maternal report: Warmth, Control, 
and Parenting Stress. The 18-month follow-up also included a measure of the emotional support 
and cognitive stimulation available to the child in the home environment, based on a combination 
of questions asked of the mother and ratings completed by the interviewer. This measure, the 
Home Observation for Measurement of the Environment-Short Form (or HOME-SF), provided a 
total score as well as Emotional Support, Cognitive Stimulation, Harsh Discipline, and Physical 
Environment subscale scores. 
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Table 1 lists the parenting measures available for the sample of the New Chance Obser- 
vational Study and the source of each measure. It is important to note that the parenting measures 
included in the New Chance Observational Study rely on several different informants (mothers, 
interviewers, and coders of the observational session videotapes). Thus, we are not confined to 
one data source in examining the New Chance Program’s possible impacts on parenting behav- 
ior. Each of the measures (along with information about its previous use and psychometric prop- 
erties) is described in detail in the monograph. 

The focal children’s development was assessed as part of the 42-month follow-up. Direct 
assessments of their cognitive development were carried out using the School Readiness Com- 
ponent of the Bracken Basic Concept Scale, which assesses children’s knowledge of such con- 
cepts as colors, letters, numbers, shapes, counting, and making comparisons. In addition, mothers 
rated their children’s behavior problems (using the Behavior Problems Index) and positive social 
behaviors (using the Positive Behavior Index) and responded to questions concerning their chil- 
dren’s health. The Behavior Problems Index provides a total score and subscale scores for be- 
haviors that reflect antisocial, anxious/depressed, headstrong, hyperactive, dependent, and peer 
conflict/withdrawal behaviors. The Positive Behavior Index provides a total score and subscale 
scores for compliance, social competence, and autonomy. 

For those children already in a formal child care situation or school setting, teachers were 
asked to complete a survey, in which they rated children’s academic and behavioral adjustment 
to school. They also completed the Behavior Problems Index and the Positive Behavior Index. 



IV. The Findings in Brief 

• Mothers in the New Chance Observational Study sample were at par- 
ticularly high risk in terms of parenting behavior. 

Some of the parenting measures used in the New Chance Observational Study had been 
used in other studies of high-risk families. On these measures, New Chance mothers had less 
positive scores than mothers in the other high-risk samples. When reading to their children, New 
Chance mothers showed lower frequencies of “Nonimmediate Utterances,” that is, speech that 
extends beyond the information in the book (the specific words in the text and the pictures 
shown) to make connections with other experiences and information. Previous research shows 
Nonimmediate Utterances to be important in laying the groundwork for later literacy in children. 
Mothers in the New Chance Observational Study also expressed hostility to their children more 
often than the mothers in another high-risk sample. Harsh mother-child interaction is important 
to development, predicting less positive adjustment in children. 

® Despite their similar economic circumstances and backgrounds, the 
mothers in the New Chance Observational Study showed variation in 
their parenting behaviors, and parenting behaviors were meaningfully 
related to the mothers’ background characteristics. 

Among mothers in the New Chance Observational Study sample, more emotionally sup- 
portive and cognitively stimulating parenting behaviors were associated with higher maternal 
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Table 1 



Parenting Measures 

Included in the New Chance Observational Study 







Time of Collection in New Chance Study 




Type of 

Parenting Measure 


Variables 


Observational 

Session 


1 8-Month 
Interview 


21 -Month 
Interview 


Chapter 


Observational 
Measures of Mother- 
Child Interaction 
Related to Literacy 


Book Reading Task: 

Total Number of 
Utterances 

Number of Nonimmediate 
Utterances 

Percentage of Immediate 
Utterances 

Number of Discussion 
Topics 

Book Reading Quality 


Derived from 
two tasks 
only: book 
reading task 
and wheels 
task 






5 




Wheels Task: 












Objects Named 
Objects/Elicitations 
Mother’s Ease of Ideas 










Observational 
Measures of 
Affective Quality of 
Mother-Child 
Interaction 


Ratings of Mother: 

Supportive Presence 

Intrusiveness 

Hostility 

Quality of Instruction 
Confidence 
Harsh Treatment 


Derived from 
full 

videotape 






4 




Ratings of Child: 












Persistence 
Enthusiasm 
Negativity 
Compliance 
Experience of Session 
Affection to Mother 
Avoidance of Mother 












Ratings of Dyadic Behavior: 












Quality of Relationship 
Boundary Dissolution 










Combination of 
Interviewer Ratings 
and Maternal Report: 
Home Observation 
for Measurement of 
the Environment — 
Short Form 
(HOME-SF) 


Emotional Support 
Cognitive Stimulation 
Physical Environment 
Harsh Discipline 
HOME Total 




X 




6 


Maternal Report 
Scales 


Warmth 

Control 




X 

X 




6 




Stress 




X 








Overall Parenting Time 






X 






Parenting Chore Time 






X 
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literacy, more educational attainment, better maternal psychological well-being, greater social 
support, and the child’s having participated in child care. 

• The New Chance Program was able to bring about positive changes in 
parenting behavior, even in a population burdened by economic stress 
and other serious difficulties. 

Positive program impacts, although modest in magnitude, were seen in both the affective 
quality of mother-child interaction and the cognitive stimulation that the mothers provided. In 
terms of the affective quality of interaction, mothers in the experimental group had significantly 
lower scores on the Harsh Treatment measure, higher scores on the HOME-SF Emotional Sup- 
port subscale, and higher scores on the Maternal Warmth scale. In terms of cognitive stimulation, 
mothers in the experimental group received higher scores on the Book Reading Quality measure. 
Positive program impacts were also found on the HOME-SF total score and on time use meas- 
ures devoted to parenting (both Overall Parenting Time and Parenting Chore Time — time spent 
engaged in such chores as feeding and bathing children). 

• Positive program impacts were found across parenting measures ob- 
tained in several different ways and from different informants. 

Differences were found for parenting measures based on direct observation of mother- 
child interaction ( Harsh Treatment and Book Reading Quality), measures based entirely on ma- 
ternal report (Warmth, Overall Parenting Time, and Parenting Chore Time), and measures that 
rely on a combination of maternal report and interviewer ratings (the HOME-SF total score and 
Emotional Support subscale). 

• Parenting behavior was an important predictor of specific child outcomes 
in this sample, as were variables reflecting maternal psychological well- 
being and the families’ larger social context. 

Positive implications of supportive and stimulating parenting behavior combined with the 
negative implications of maternal psychological distress and stress in the larger social context to 
shape the developmental outcomes of children in this sample. 

• Modest improvements in parenting behavior, in this context, did not suf- 
fice to bring about positive program impacts on child outcomes. 

Given the high levels of maternal psychological distress and stress in this sample, bring- 
ing about positive impacts on child outcomes might have required (1) a more intensive “dosage” 
of parenting education classes or other program components that enhance parenting behavior, (2) 
sufficiently intense program components directly focusing on the mothers’ psychological well- 
being and living situations, and/or (3) high-quality child care, with children participating over a 
sustained period of time. 

• Observational measures add to the understanding of parenting behavior 
within this sample in multiple ways. 

Observational measures increase one’s confidence in findings on program impacts be- 
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cause the coding is carried out in an extremely rigorous way. They also tap into certain behaviors 
that are important to children’s development, but that mothers may not be able to report on (such 
as their use of Nonimmediate Utterances during book reading). Also, the findings indicate that 
observational measures have advantages over interview-based measures for predicting variation 
in children’s scores on specific child outcomes. For example, consideration of the observational 
measures of mother-child interaction significantly improved prediction of the Behavior Problems 
Index (as reported on by the mother) even when parenting measures based on interviews had al- 
ready been taken into account. In short, observational measures provide valuable information that 
can enrich the evaluation of programs such as New Chance. 



V. Parenting Behavior in This Sample Relative to Other Samples 

The observational study interaction tasks, and the coding of behavior from those tasks, 
grew out of previous research in the laboratories of Catherine Snow, Jeanne De Temple, and their 
colleagues at Harvard University, and Byron Egeland, Nancy Weinfield, John Ogawa, and their 
colleagues at the University of Minnesota. Since the measures have been refined and improved 
over time, it is generally not possible to compare directly the findings from the New Chance Ob- 
servational Study with those from previous studies. For a few specific measures, however, modi- 
fications in rating scales have not occurred. When we look at findings for those measures that can 
be compared across studies, we see that New Chance Observational Study families are at par- 
ticularly high risk in terms of parenting behavior. 

One key measure of mother-child interaction related to the development of children’s lit- 
eracy is the proportion of maternal talk during the book reading task that involves Nonimmediate 
Utterances, that is, connecting the story and pictures to other events, people, and objects. Such 
talk also involves making predictions, asking for inferences, and providing explanations. Previ- 
ous research using this measure in the Home-School Study of Language and Literacy Develop- 
ment, a longitudinal study of low-income mothers and their children, found maternal use of 
Nonimmediate Utterances to be related to important child outcomes in the early years of ele- 
mentary school, especially the children’s use of language and their literacy skills. 

Nonimmediate Utterances in the context of a book reading task constituted 10.5 percent 
of mothers’ utterances in the Home-School Study of Language and Literacy Development, but 
only 3.5 percent of talk among New Chance Observational Study mothers. The researchers note 
that “Since we have found that maternal use of Nonimmediate Talk relates to later child out- 
comes ... the very low proportion of Nonimmediate Talk produced by the New Chance mothers 
is troubling.” 

The ratings scales of the affective quality of mother-child interaction used in the New 
Chance Observational Study are adaptations of scales developed in the Minnesota Mother-Child 
Project, a longitudinal study of high-risk mothers and children. The Hostility rating scale has not 
been substantially modified for the present study, and results can be compared across studies. 
Ratings of 5 or above indicate that a mother is more hostile than not in interacting with her child 
during the course of the mother-child tasks. Seven percent of the New Chance Observational 
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Study sample scored 5 or above compared with 2.7 percent on the Minnesota Mother-Child Proj- 
ect sample. 

These findings suggest that the New Chance Observational Study sample is at greater risk 
in terms of parenting behavior than the previously studied high-risk samples, underscoring the 
importance of attempting to enhance parenting behavior in the population of young single moth- 
ers in poverty. 



VI. Variation in Parenting Behavior in Light of Maternai and Family 

Characteristics 

Family characteristics were significantly linked with parenting behaviors, and associa- 
tions were more consistent with 18-month than baseline variables, suggesting that the more cur- 
rent family context is of greater importance to parenting behaviors. These characteristics in- 
cluded social support; measures of the mother’s psychological well-being, residence pattern, 
maternal education, and literacy; and the child’s participation in child care during the initial 18- 
month follow-up period. 

For example, mothers who reported more sources of social support at the time of the 18- 
month follow-up interview had more positive relations with their children in terms of the ob- 
served affective quality of mother-child interaction, the observed literacy-related aspects of inter- 
action, and the harshness of discipline as measured by the HOME-SF. Mothers at high risk for 
depression at the 18-month follow-up had lower scores on all but one of the HOME-SF 
subscales, had lower observed Book Reading Quality scores, reported more parenting stress, and 
described themselves as using more controlling disciplinary practices. 



VII. Program Impacts on Parenting Behavior 

Significant positive program impacts on parenting behavior were found on a range of 
parenting measures: 

• Measures based on direct observation of mother-child interaction: 

Harsh Treatment 
Book Reading Quality 

• Measures based on maternal report alone: 

Maternal Warmth 
Overall Parenting Time 
Parenting Chore Time 

• Measures based on a combination of maternal report and interviewer ratings: 

HOME-SF total score 
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HOME-SF Emotional Support subscale 



Positive program impacts occur in aspects of parenting that have previously been identi- 
fied as particularly important to the development of children in poverty. For example, harsh par- 
ent-child interaction has been found to occur with greater frequency among families experiencing 
economic hardship. Harsh parent-child interaction, in turn, is a key contributor to the less favor- 
able adjustment of children in poverty. 

In general, the findings indicate that the New Chance Program had positive impacts on 
parenting behaviors important to development. There are three caveats, however. First, there 
were differences on only a minority of the parenting measures examined. Second, all but one of 
the program impacts, that on the HOME-SF Emotional Support subscale, were small in magni- 
tude (“effect size”)- Finally, a single significant finding ran counter to this pattern of positive im- 
pacts: Mothers in the experimental group received lower Ease of Ideas scores, observed during 
the mother-child task that called for eliciting from the child the names of objects that have 
wheels. That is, mothers in the experimental group were observed to be less facile in coming up 
with hints and clues for the child. This difference, however, was found to be attributable entirely 
to a larger number of mothers within the experimental group who did not grasp the goal of this 
task at all. Thus, this group difference might just as easily be interpreted as an indication of task 
or test anxiety, or of problems with the interviewers’ explanations of this task, than as a reflec- 
tion on the quality of the mothers’ parenting behavior. 

The research team concluded that the New Chance Program had positive, albeit modest, 
impacts on parenting behavior. 



VIII. Program Components That Contributed to Positive Program Impacts om 

Parenting Behavior 

Among mothers in the experimental group within the New Chance Observational Study 
sample, greater participation in parenting education classes was related to more positive parent- 
ing behavior. Interestingly, however, this pattern was also found for participation in other New 
Chance Program components. 

We looked at four aspects of experimental group mothers’ participation in New Chance: 
(1) participation in parenting education classes, (2) participation in a broader set of program ac- 
tivities that addressed parenting behavior in some way, (3) participation in human capital devel- 
opment components of the program (that is, components intended to improve the mothers’ ability 
to obtain and keep a job, including adult education and employability development classes), and 
(4) total program participation. 

Even after controlling for baseline characteristics that predicted each of these aspects of 
program participation, certain parenting measures continued to be significantly associated with 
the extent of program participation among mothers in the experimental group. Thus, within this 
group, scores on the HOME-SF Emotional Support subscale continued to differ significantly ac- 
cording to extent of participation as defined in each of the four ways; and the observational 
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measure of Book Reading Quality differed according to participation in the human capital devel- 
opment components of the program as well as in the program overall. 

It should be noted that characteristics we did not have baseline measures of, and thus 
could not control for, might be linked both to greater program participation and to parenting be- 
havior, and these undocumented variables might help account for the associations we have noted. 
These findings nevertheless raise the possibility that components of programs for young mothers 
in poverty including but going beyond parenting education may have positive implications for 
parenting behavior. Further study is needed to explore how and why program components di- 
rected at mothers’ education or employment skills, and their overall program participation, might 
affect their parenting behavior. 



IX. Parenting Behavior as a Predictor of Child Outcomes 

The findings on child outcomes in the New Chance Demonstration present us with a 
paradox. Although there were positive impacts on measures of parenting, children in the experi- 
mental group did not do better in terms of their cognitive and social development when these 
were assessed at the 42-month follow-up. On most measures, there were no program impacts. 
However, on some measures there were unanticipated negative program impacts. For example, 
mothers in the experimental group described their children as having less positive social behav- 
ior. This pattern held in both the full demonstration study sample and the smaller observational 
study sample. 

Clearly, we need to consider factors other than parenting behavior as contributors to chil- 
dren’s development — for example, mothers’ psychological well-being, family economic status, 
and the experiences of mothers and children outside the mother-child relationship. New Chance 
appears to have had unexpected negative program impacts on some of these further factors. 4 For 
example, mothers in the experimental group in the observational study sample were more, rather 
than less, depressed at the time of the first (18-month) follow-up survey. Mothers in the experi- 
mental group of the observational study sample also reported less life satisfaction. 

We considered both the role of parenting behavior and variables that measured other im- 
portant aspects of the mothers’ and children’s lives as predictors of five selected child outcomes 
within the observational study sample (children’s total scores on the School Readiness Compo- 
nent of the Bracken Basic Concept Scale, the Behavior Problems Index total score as reported by 
both mother and teacher, and the Positive Behavior Index total score as reported by both mother 
and teacher). We confirmed that when measures of parenting behavior were taken into account, 
our ability to predict children’s scores on several of these outcomes improved significantly 
(specifically, the maternal report of the Behavior Problems Index and Positive Behavior Index 
and the teacher report of the Positive Behavior Index). 



4 See Janet C. Quint et al., New Chance: Interim Findings on a Comprehensive Program for Disadvantaged 
Young Mothers and Their Children (New York: MDRC, 1994), Chapter 6, and Quint, Bos, and Polit, 1997, Chapter 
6 . 
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More supportive and stimulating parenting thus predicted more optimal developmental 
outcomes on specific measures of development. However, in addition, we found that variables 
reflecting maternal psychological well-being (for example, measures of life satisfaction and of 
aggravation and stress in parenting) and variables reflecting the larger social context of the fami- 
lies (for example, measures of difficult life circumstances and of number of changes of residence 
since enrolling in the evaluation) were also significant predictors of several child outcomes. 
Greater maternal psychological distress and greater stress in the larger social context predicted 
less positive developmental outcomes. 

An important finding of this study, then, is that while parenting behavior was a signifi- 
cant predictor of specific child outcomes, it was not the only predictor. The positive influence of 
supportive and stimulating parenting behavior combined with negative influences of psychologi- 
cal distress in the mother and stress in the family’s larger social environment. Children’s devel- 
opmental outcomes reflect influences not only from within but also outside the mother-child 
dyad. 

Outcomes for children in the context of an intervention such as New Chance might im- 
prove if the intervention directly addressed these problems through more intensive mental health 
intervention for the mothers; if the “dosage” of program components with positive implications 
for children’s development, such as parenting education, were substantially increased; or if pro- 
gram elements targeted to the children themselves were strengthened. Direct observations of 
New Chance child care settings placed these child care centers just below the “good” range in 
terms of quality. 5 For positive child outcomes to occur, children from high-risk families may 
need higher-quality care. In addition, beyond the first program follow-up (at 18 months), children 
in the New Chance experimental group were not found to participate in more child care than 
those in the control group. Indeed, the increased child care participation of children in the ex- 
perimental group tended to occur only during the first months of their mothers’ program partici- 
pation, when the mothers were engaged in on-site classes and activities. Positive program im- 
pacts on children may require sustained participation in high-quality child care. 



X. How Observational! Measures of Parenting Contribute to a Study Such as the 

Evaluation of New Chamce 

Measures of parenting based on direct observation of mother-child interaction contribute 
to our understanding of the New Chance Program in several ways. First, these measures increase 
our certainty about program impacts on parenting. If positive impacts on parenting were found 
only for maternal report measures, we might question them on the grounds that mothers in the 
experimental group (aware that they had access to New Chance Program services and that out- 
comes of the program were being assessed) could report more favorably because they felt it to be 
expected of them. Even interviewers aware of which research group a family was in could be 
subject to such “response biases.” Coders of the videotapes were unaware of which research 
group a family was in, yet positive program impacts were found on observational measures. 



5 Quint et al., 1994, pp. 73-76. 
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Second, observational measures provided information that was different from and com- 
plementary to that provided by interview-based measures of parenting. For example, a positive 
program impact was found on the observational measure of Book Reading Quality, yet no pro- 
gram impacts were found on the interview-based measure of cognitive stimulation (the HOME- 
SF Cognitive Stimulation subscale). The observational measure looks at the nature or quality of 
mother-child interaction in a book reading context, for example, the mother’s fluency, intonation, 
and comfort level in reading the book to her child. By contrast, the HOME-SF measure of cogni- 
tive stimulation assesses the quantity of literacy-related and other stimulating activities: for ex- 
ample, how many books were in the home, and how often the mother reads to the child. Without 
a measure of the nature or quality of cognitive stimulation, we might not have known that there 
were program impacts in this area. Correlations between observational and interview-based 
measures of parenting confirm that these measures are related but do not substantially overlap. 

Finally, we found the observational measures to be helpful in predicting child outcomes. 
When we distinguished among maternal report scales, HOME-SF subscales, and observational 
measures as predictors of the selected child outcomes, we found that the HOME subscales and 
observational measures were generally better predictors than the maternal report scales. We 
also asked whether observational measures added to our ability to predict variation in children’s 
developmental outcomes after the maternal report scales and HOME-SF subscales were already 
taken into account. We found that for two of the five selected child outcomes, observational 
measures added significantly to our ability to predict child outcomes even when the other par- 
enting measures had been taken into consideration. 

The fact that the observational measures do not rely at all on maternal report also helps 
eliminate the possibility that associations between parenting behavior and those child outcomes 
based on maternal report (for example, the Behavior Problems Index, as reported by the mother) 
are not merely a reflection of common attitudinal or reporting tendencies across different sets of 
maternal report measures. 

Thus, observational measures of parenting behavior provide different information than 
that available through interview-based measures of parenting, diminish concerns about possible 
response biases and correlated measurement error, and add significantly to the ability to predict 
specific child outcomes. Where the examination of parenting behavior is a high priority, obser- 
vational measures of parenting add substantially to the strength of an evaluation. 
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Part II: Methodological Assessment 
of the New Chance Observational Study 



I. The Methodological Context for the New Chance Observational Study 

The New Chance Observational Study lies at the confluence of rising interest in policy- 
relevant research among developmental psychologists, interventions focusing on two generations 
in a family, and increasing demand for nontraditional forms of survey research. In attempting to 
increase the size and representativeness of samples, some developmentalists are departing from 
the familiar model of laboratory-based research, sometimes supplemented with home visits, to 
undertake their data collection through contracted survey research. Survey research on parenting 
and child development has traditionally involved querying parents about their parenting practices 
and their children’s development. In recent years, survey interviewers have also been asked to rate 
parent-child interactions and the home environment, and, in a few cases, to administer structured 
parent-child interactions similar to those usually carried out in the child development laboratory. 
New Chance, while not alone in its use of survey methods to study child development, is one of the 
few studies to use survey interviewers to conduct observational work with mothers and children. 
Within this context, Part II of the monograph seeks to familiarize readers with the “survey 
model,” document how the observational study was conducted, assess the success of the effort, 
and consider both specific recommendations for future work and broader implications for re- 
search design. 

This study affords a unique opportunity for examining methodological issues in the 
measurement of parenting and child outcomes for two reasons. First, it taps a diverse set of 
measures and data sources: taped mother-child interactions coded under rigorous conditions in 
university laboratories, self-reports of mothers, and ratings by survey interviewers of mother- 
child interactions and the home environment. Just as important as the rich variety of domains 
tapped is the fact that the study was conducted within the framework of a survey research model, 
while “stretching” this model to exploit it in innovative ways. 



II. Implications of Conducting Research Within a Survey Model 

Chapter 10 of the monograph seeks to enhance the value and accessibility of the survey 
field to social scientists who have not made survey research the primary focus of their careers 
and to begin to bridge the communication gap between survey researchers and their colleagues 
— even from the same disciplines — who may emphasize a substantive research agenda more 
than a method. Without a full appreciation of the constraints of the survey model, researchers 
who commission survey work may have difficulty managing it and even find themselves disap- 
pointed with the results. At the same time, they will be shortchanged if they look to the field 
primarily for a data collection capability and fail to heed its lessons on survey-based measure- 
ment. 



In contrast to an academically oriented model, survey research is built on a division of labor 
between those responsible for the conceptual and analytical aspects of research and those who actu- 
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ally collect the data. The survey model places responsibility for data collection in the hands of 
“distant proxies” for “absent researchers.” These proxies, the survey interviewers who collect data 
on surveys ranging from small local studies to large recurring government surveys, are neither as- 
sumed to possess nor expected to master a conceptual appreciation of the research in which they 
participate. For these reasons, survey researchers provide interviewers with explicit rules to di- 
rect them through the data collection — rules that, in theory at least, require minimal judgment to 
apply. The resulting survey model is characterized by an orientation toward production, a con- 
trived and stylized format for the interaction of interviewer and respondent, and precise pro- 
gramming and standardization of interviewer behaviors. Designing sound data collection instru- 
ments and procedures involves accommodating to these three aspects of the survey model. 



III. Turning to the Survey Field for Help with Measurement Issues 

The constraints, or “rules,” of the survey model may be viewed as potential limitations in 
conducting social science research, but the survey community’s trove of research on survey 
measurement can be viewed as a valuable resource. The underlying focus of most of this work is 
the reduction of error in surveys. 

The bulk of the measurement literature has focused on the design of measurement in- 
struments, that is, questionnaires, and less on other aspects of measurement, such as interviewer 
or mode effects. If this literature yields an overarching lesson, it is a humbling one: that there are 
many, many features of questions and questionnaires that can affect survey response. An aware- 
ness of such measurement threats is helpful both in developing measures and related data collec- 
tion instruments and in assessing their likely strengths and limitations. 

The survey literature also speaks to the ways in which interviewers may affect measure- 
ment in a study like this. Evidence for the prominence of interviewer effects in situations requir- 
ing more judgment or unprogrammed behavior is especially relevant to the present work, which 
required interviewers to make substantive ratings of the home environment and mother-child in- 
teraction, obtain time-use information through a series of open-ended probes, and administer a 
scripted observational protocol, while applying general principles to unscripted situations. 

Given the importance of interviewers and the expansion of their responsibilities with ever 
more challenging studies, focusing on the cognitive demands of their tasks will help in assessing 
what is sensible to ask them to do and helping them to do it. The last decade or so has seen the 
emergence of a broader interest in cognitive aspects of survey response, beyond simply the recall 
of information. This interest has been directed primarily toward the tasks facing respondents during 
an interview, such as the strategies they use to estimate and report the frequency of a particular be- 
havior. The cognitive framework can also be applied to the cognitive tasks faced by interviewers 
during a survey interaction, by examining, for example, the ability of interviewers to process the 
information necessary for making substantive ratings at the same time they are reading interview 
questions and recording answers. Viewing the demands of the survey interaction on both respon- 
dents and interviewers from a cognitive perspective provides a helpful framework for integrating 
what is known about the instruments used to collect data, the interviewers who administer them, 
and the respondents who provide the data. 
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IV. Steps in Implementimg the New Chance Observational Study 



As discussed in Chapter 1 1 , and summarized below, implementing the observational re- 
search that is the central subject of this monograph involved both the design of data collection 
instruments and procedures and the data collection effort itself. In carrying out these tasks, we 
confronted and sought to address a host of survey measurement issues, most notably the over- 
arching issue of adherence to the goals and objectives of the “absent researchers” who place their 
research in the hands of survey interviewers. Our approach included the following steps. 

o Tasks developed in university laboratories were adapted for survey ad- 
ministration by strictly scripting them in tbe format of a structured sur- • 
vey questionnaire to ensure standardized delivery by survey interviewers. 

Survey interviewers are not expected to have the appreciation of a study’s theoretical un- 
derpinnings that would allow them to work from only a researcher’s semistructured outline. 
Therefore, the strategy for realizing the objectives of the “absent researchers” was to program 
interviewer behavior in the observational session carefully. This meant providing interviewers 
with a data collection instrument resembling a survey questionnaire, which specified the precise 
language to be used and the actions to be taken. The instrument included detailed instructions for 
presenting, arranging, and withdrawing the various props used in the tasks (book, games, and 
gift) and for coordinating management of the props with the script. Also included were language 
and instructions for verifying that the mothers understood the objectives of the task, decision 
rules for determining how much time to spend on each, and, for one task, decision rules as to 
whether a simple or complex version, or both versions, of the task was to be administered. 

© The potentially unlimited number of situations that could not be scripted 
explicitly were addressed by developing a set of general principles that 
survey interviewers were expected to apply. 

A survey questionnaire typically tries to anticipate and provide for every scenario that can 
be tapped by a question; this is generally done by creating response categories for recording an- 
swers to a question that are exhaustive and (usually) mutually exclusive and by employing “skip 
patterns,” in which questions are asked or not depending on the answers to prior questions or 
characteristics of the respondent. Like response categories, skip patterns must be designed to an- 
ticipate all of the situations to be encountered. 

By contrast, the data collection instrument for the observational study could not anticipate 
the virtually infinite range of behaviors and events that might occur during the session, not only 
involving the mother and child but also other people and events in the household. Since it would 
have been impractical to embed instructions for responding to even a sampling of hypothetical 
situations in the data collection instrument, a separate set of guiding principles, rules, and exam- 
ples was developed, which interviewers were expected to apply in the actual data collection 
situation. Although our intent was to minimize the need for judgment on the part of the inter- 
viewers, they did have to label a situation correctly in order to respond appropriately, which in- 
creased the complexity of the task. For instance, they needed to understand the distinction be- 
tween a mother’s request for clarification of the instructions or objectives of a task and a request 
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for guidance as to how to work with her child, since these two situations called for quite different 
responses. 

• Although the activities that constituted the observational session had en- 
joyed extensive use in laboratory-based research, pretesting was neces- 
sary to investigate issues specific to survey administration. 

Two small pretests were conducted to help in refining the adaptation of the laboratory 
protocols for survey administration, check on the effectiveness of the tasks among young disad- 
vantaged respondents, and investigate a number of practical implementation issues. Even these 
small pretests anticipated a variety of situations and issues that would arise again in the main 
study. For instance, the home environments in which the data collection occurred were typically 
characterized by a host of distractions and interruptions from television and radio, children, and 
other sources. The pretests also revealed how the interviewers’ personal styles could affect the 
way in which the sessions were administered. For instance, some interviewers seemed to have 
particular difficulty maintaining a neutral, nondirective stance. Making sure the mother under- 
stood each task first arose as a concern during the pretesting and remained an issue during the 
data collection, since interviewers were charged with the somewhat incompatible goals of clari- 
fying the objectives of the task if the mother failed to understand it, without telling the mother 
how she should work with her child. Requirements that interviewers make subtle distinctions — 
reinforce effort rather than performance, for example — posed problems during the pretests and 
continued to do so during the study. Many other issues were successfully resolved through pre- 
testing, however, and rarely resurfaced. 

• Training materials and curricula, developed through the design and pre- 
testing process and supplemented by ongoing quality control, provided 
the critical link between the researchers designing the work and the lay 
survey interviewers collecting the data. 

Training materials . The basic resources included a procedures manual for the study and 
a training videotape, which supplemented training that interviewers had already received for the 
New Chance 1 8-month survey to make the HOME-SF (Home Observation for Measurement of 
the Environment-Short Form) and related ratings also called for by the observational study. 

The procedures manual developed for the study was similar in many ways to manuals 
used for more typical surveys, covering a mix of substantive and administrative requirements and 
emphasizing use of the data collection instrument. The manual also addressed topics specific to 
the observational study such as the ways in which the assignment resembled and differed from 
traditional survey interviewing, special requirements for conducting the observational session, 
principles for relating to the child during the visit, definitions of questionnaire items not covered 
in training materials for the 1 8-month survey, detailed guidelines for handling unscripted situa- 
tions, coordination in the field between the two members of the data collection team (interviewer 
and videographer), and technical instructions related to the taping. The manual was designed not 
only for training the data collection teams but also to serve as a reference resource throughout the 
study. 
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Training curriculum . The first component of interviewer training consisted of comple- 
tion of a home-study package. Trainees were expected to study the procedures manual and re- 
view a sample videotape. In addition, interviewers were expected to practice delivering the ob- 
servational script and complete a quiz covering the major requirements of the study. 

A one-day “classroom” training session followed. A joint session of interviewers and 
videographers began with an overview of the project, followed by a critical viewing of the train- 
ing videotape, with discussion of the points it illustrated. In the next module, interviewers and 
videographers practiced working together in assembling and removing the various props. The 
joint session concluded with a discussion by the group of the need and mechanisms for close 
communication between interviewers and videographers. 

In a separate session later in the day, videographers were introduced to the specific 
equipment to be used in the study and received training on their technical responsibilities. A si- 
multaneous session for interviewers began with a review of the maternal self-report items used in 
the observational study questionnaire but not in the 18-month survey, which were to be com- 
pleted during the interview segment of the observational session. This was followed by a review 
of the quizzes completed as part of home study. Then, with trainers playing the parts of mothers 
and children, each interviewer administered two “mock” observational sessions that had been 
scripted in advance to include a variety of situations for them to handle. 

After the classroom session and as the last step in the training, each interviewer com- 
pleted a “real” practice observational session with a family in the community similar to those in 
the New Chance sample. Interviewers and videographers received feedback after review of their 
work by members of the observational study team before beginning work with actual sample 
members. 



V. Findings on the Administration of the Observational Sessions 

In reflecting on this work, the challenges that faced the data collection teams and the per- 
formance of survey interviewers in a nontraditional role were of particular interest. Interviewers 
were to administer the protocol in such a way that all the mothers began with a clear idea of the 
objectives of each task and then maintain a polite professionalism and neutral stance while each 
dyad worked together. The challenges to mothers and children of working together on tasks that 
required some “stretch” for the children were not to be magnified by problems created by the in- 
terviewer, such as failing to explain a task adequately or creating additional anxiety by empha- 
sizing the child’s performance. Nor were the inherent challenges of the tasks to be reduced by 
“helpful” suggestions from the interviewer. Other extraneous influences that we had hoped to 
minimize were intrusions by other people or events during the observational session. 

Chapter 12 of the monograph presents findings on how the observational sessions were 
actually administered. These findings draw on a variety of data sources: interviewer ratings, ma- 
ternal self-reports, comments about the session and its administration recorded by the university 
coders, and variables coded in the university laboratories. 
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• The ideal environment for conducting the sessions was rarely encoun- 
tered. The sessions were typically conducted with persons besides the 
mother and child present, and interruptions and distractions were com- 
monplace. 

While interviewers tried to schedule the sessions when only the mother and the focal 
child were at home, they were generally unable to do so: in 70.9 percent of the cases, there was 
someone else present. Children other than the focal child were most common, being present in 
51.0 percent of the cases. Although the presence of others did not guarantee interruptions of the 
session, this was often the effect. In 33.8 percent of the sessions another child or children inter- 
rupted the session at least once; in 7.6 percent of the sessions there were three or more such inter- 
ruptions. In addition, background noise was often present: in 46.6 percent of the cases there was 
audible background conversation, while a radio, television, or stereo contributed background 
noise in 29.3 percent of the cases. 

• A proxy measure of interviewer performance, representing overall com- 
petence in administering the observational session, suggests that a large 
majority of the sessions were administered satisfactorily, if not optimally. 

Comments from the Harvard University laboratory on the two tasks coded there — the 
book reading and discussion activity and the wheels task — served as a proxy for overall inter- 
viewer performance. Based on these comments, we conclude that there was certainly room for 
improvement in the interviewers’ performance: in 42.0 percent of the sessions, the coders noted 
some deviation from the protocol. However, in about half of these cases, only a relatively minor 
deviation on one of the two tasks was noted. Thus, overall, 79.9 percent of the sessions were 
characterized by either no deviations or only a relatively minor deviation on one of the two tasks. 
Still, a deviation judged to be at least moderately serious was noted in about 10 percent of ad- 
ministrations of each task. Many of the deviations cited by the coders clearly represent errors by 
the interviewers, in that they disregarded instructions or principles contained in the various study 
materials and addressed in interviewer training. The most common of these, seen in 8.9 percent 
of the sessions for the book activity and 7.2 percent for the wheels task, involved offering direc- 
tion, intervening in the dyad’s work, or otherwise abandoning a neutral stance in some way. 
Other problems arose either because interviewers were expected to make difficult judgments in 
the press of the situation or because they were discouraged from exercising their own judgment, 
sometimes resulting in behavior that appeared awkward or inappropriate when viewed on tape. 

• Considerable variation was observed in the proficiency with which the 
survey interviewers administered the sessions, with the more proficient 
conducting more sessions. 

The number of sessions completed by each interviewer ranged from 1 to 52. There was a 
positive association between interviewer proficiency and the volume of cases completed, with 
more proficient interviewers completing more cases. This finding may have resulted from re- 
training and practice effects or the selection out of the less proficient (or less committed) inter- 
viewers. 
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® Despite concern about the potential intrnsiveness of a study like this, 
coupled with the mild stress created by the challenge of the tasks, most 
mothers found the session to be a positive experience. 

Examination of the mothers’ self-reported experience of the observational session re- 
vealed that 54.1 percent rated it in the very positive range on a composite measure, with virtually 
all of the rest rating it in the intermediate range. But when focusing only on mothers’ perceptions 
of the more challenging performance-related aspects of the experience (for example, nervous- 
ness, difficulty working with her child), 37.9 percent reported a very positive experience and 
14.6 percent expressed quite negative feelings. 

• The age of the focal child was the central mediator off the experience of 
the session for both mother and child and was related to the frequency of 
interviewer errors as well. 

While the various indicators of the experience of the session that were examined were 
associated with one another, the child’s age was the common thread that united them, with the 
experience being more negative the younger the child. The interviewer’s difficulty in conducting 
the session also apparently related to the age of the child, with interviewer errors being more fre- 
quent the younger the child. 



VI. Methodological Implications of the New Chance Observational Study 

We have asked whether it is possible to carry out observational work within a contracted 
survey research model and, in Chapter 13 of the monograph, answer the question with a qualified 
“yes.” We argue for an approach built on explicit recognition of the differences between the 
backgrounds of survey interviewers and developmental psychology researchers, as well as rec- 
ognition of the cognitive demands of this work. The cognitive processes employed by interview- 
ers in responding to unprogrammed situations, in which they must exercise judgment because the 
situations cannot be precisely scripted, are of particular interest. To optimize the value of the data 
collected through survey methods, it is necessary to recognize the complexity of the inter- 
viewer’s assignment and to pay careful attention both to the design and testing of data collection 
protocols and to the selection, training, and supervision of interviewers. Careful design efforts 
should be followed by a formal interviewer certification process, with ongoing quality control 
and feedback to interviewers on their performance. 

But interviewer effects represent an ever present source of potential bias despite efforts 
undertaken to minimize them. This is true for all measures with which they are associated, 
whether derived from interviews, interviewer ratings, or administration of an observational pro- 
tocol. To measure the extent of such effects, random assignment of (fairly large numbers of) 
cases to interviewers is necessary, but difficult and costly to accomplish. However, it is possible 
to control for interviewer effects in a study employing an experimental design, by ensuring that 
each interviewer’s caseload mirrors the distribution of experimentals and controls in the overall 
sample. 
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VII. Implications for Measurement Strategies in Studying Parenting and 

Child Outcomes 

Researchers faced with choices about the measures and methods to employ in studying 
parenting and child outcomes require a framework for making decisions about study design. As 
discussed in Chapter 14 of the monograph, such a framework can encompass both methodologi- 
cal and substantive considerations. Methodological considerations include feasibility and data 
quality. Based on the New Chance Observational Study, as well as other work, we conclude that 
developmentalists can draw on a methodological repertoire within a survey framework that in- 
cludes maternal self-reports on parenting, interviewer ratings, cognitive assessments of children, 
interviews with children, and administration of observational protocols. The availability of such 
a broad range of measurement techniques means that researchers can assemble a “balanced port- 
folio” of measures that spread the measurement “risk” across methods having different strengths 
and vulnerabilities. 

The overarching substantive consideration in asking whether to include observational 
work within this portfolio is the “value added” of the observational data. Such work could be un- 
dertaken as part of a freestanding study, in which all participants provide both self-reported data, 
perhaps enhanced by interviewer observations, and observational data. Or observational work 
could be embedded within a larger survey effort, as was the case in this study. In considering the 
embedded model, the “analytical leverage” provided by the observational data is a consideration. 
If the subsample participating in the observational component of the study is selected randomly, 
it is possible to generalize to the larger sample, and/or to the population from which it was 
drawn, through statistical weighting procedures. Other potential analytical strategies include us- 
ing the richness of the observational measures to help in interpreting data from other sources, in- 
cluding assessing how much confidence to place in other measures that are potentially subject to 
different method effects. Finally, observational measures from a subsample may be used to aug- 
ment quantitative survey findings. 

Researchers, however, should not expect to find a single decision rule for determining when 
observational research within a survey model is warranted. Every study is different. Survey-based 
observational work makes sense when it serves well-articulated analytical objectives and is con- 
ceptualized as an integral part of an overall research design. If resources are available and the po- 
tential added value of observational measures is clear for a particular study, the observational 
method is a valuable adjunct to more conventional survey approaches. 
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Part I: The New Chance Observational Study 

Chapter 1 

Introduction: The Context for the New Chance Observational Study 

Martha J. Zaslow 



The New Chance Observational Study provides detailed and sensitive 
measures of mother-child interaction for a subset of the families who participated 
in the full evaluation of the New Chance Program. This observational study ad- 
dressed four broad issues: (1) whether the New Chance Program, a comprehen- 
sive and supportive intervention for teenage welfare mothers with limited educa- 
tion, had positive impacts on parenting behavior; (2) how measures of parenting 
collected by direct observation and through interviews are related, and how these 
two kinds of parenting measures differentially predict child outcomes; (3) what 
role parenting measures play, relative to such other important factors as mothers ’ 
psychological well-being and family economic self-sufficiency, in shaping the de- 
velopment of the young children of these mothers; and (4) what methodological 
issues researchers need to be aware of when they study parenting and child out- 
comes through a survey research approach that draws on interview-based meas- 
ures (maternal self-report and interviewer ratings) as well as measures based on 
direct observation of mother-child interactions. 

This first chapter of the monograph provides the context for the New 
Chance Observational Study by describing the evaluation within which this spe- 
cial study is embedded. We summarize the goals and components of the New 
Chance Intervention, as well as the results available from the full New Chance 
Evaluation. The chapter concludes by noting the specific questions that will be 
addressed in the observational study monograph and by providing an overview of 
its organization. 



I. Goals of the New Chance Observational Study 

This monograph presents the results of the New Chance Observational Study, a special 
study embedded within the evaluation of the New Chance Intervention . 1 Four broad issues were 
addressed by this study: 



1 Throughout this monograph we use the terms “New Chance Program” or “New Chance Intervention” to refer 
to the classes, counseling, and further supportive services offered to New Chance participants and their children. 
The “New Chance Evaluation” includes the assessment of program implementation, cost-benefit analysis, and as- 
sessment of both short- and longer-term program impacts on mothers and children. The “New Chance Demonstra- 
tion” encompasses both the program and its evaluation. The New Chance Intervention and Evaluation will be de- 
scribed in detail later in the chapter. 
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• whether a comprehensive intervention for young mothers who are high 
school dropouts receiving welfare improves their parenting practices, as 
measured through direct observation of mother-child interaction and 
through interview-based measures of parenting; 

• how closely measures of parenting behavior collected via direct observa- 
tion of mother-child interaction and through interviews with mothers are 
related, and how well each kind of parenting measure predicts child out- 
comes; 

• what role parenting behavior plays, relative to other important influences 
(such as maternal psychological well-being and the family’s economic 
self-sufficiency), in shaping the development of the young children of 
mothers participating in this intervention; and 

• what methodological issues are raised when researchers augment a pro- 
gram evaluation by embedding observational research in a survey re- 
search context. 

In this observational study, 290 of the 2,322 mothers in the New Chance Evaluation re- 
ceived an extra visit to their homes to collect data following the 18-month follow-up within the 
larger evaluation. The extra visit, carried out on average 21 months 2 after enrollment in the New 
Chance Demonstration, involved videotaping mothers interacting with one of their children while 
reading a book, carrying out a series of teaching tasks, and presenting a small gift to the child. 
The children videotaped interacting with their mothers were between 27 and 63 months old with 
most between 30 and 60 months. Two sets of measures were derived from coding of the video- 
tapes: measures of the affective quality of mother-child interaction, and measures of mother-child 
interaction related to the development of literacy. A brief interview with the mothers at the time 
of the observational session provided measures of mothers’ allocation of time to parenting and 
further data on such key issues as use of child care and current employment. The interview car- 
ried out 18 months after enrollment for the full evaluation sample (including the observational 
study sample) provided further interview-based measures of parenting, derived from maternal 
self-report interview items as well as interviewer ratings of the home environment. 

The New Chance Evaluation provides a unique context for asking whether a comprehen- 
sive program for young welfare mothers can affect parenting behavior, how important parenting 
behavior is relative to other factors in shaping child outcomes, and how to go about measuring 
parenting behavior within such an evaluation. Several characteristics of the New Chance Pro- 
gram distinguish it from other efforts to improve the life circumstances of mothers on welfare. 
First, the intervention itself, in several important ways, went beyond previous interventions for 
families in poverty. For example, New Chance focused on a population that, while of critical im- 



2 

The plan for the study called for the in-home data collection effort of the observational study to be carried out 
as soon as possible following completion of the 18-month follow-up, and within four months of that follow-up, 
within the larger evaluation. While the observational session occurred 21 months after random assignment, on aver- 
age, it ranged from 16 to 36 months after random assignment. 
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portance for welfare policy, is little studied: that subgroup of young mothers on welfare who 
have already dropped out of school. This group is at risk in terms of both its parenting behavior 
(with implications for the well-being of the next generation) and its long-term economic well- 
being. Second, New Chance was one of only a handful of new interventions for families in pov- 
erty that followed a “two-generational” approach (Smith, 1995b): that is, an approach that seeks 
to support the development of both parent and child, rather than focusing on one generation 
alone. 



Perhaps most important, while other interventions focus on enhancing mothers’ parenting 
behavior or their economic circumstances, New Chance sought to enhance both aspects of moth- 
ers’ lives simultaneously. The program included parenting classes and incorporated material re- 
garding parenting behavior in several further program components, such as life skills training 
classes, family planning classes, and counseling. The program also included multiple program 
components aimed at strengthening the young mothers’ employability. Mothers started with 
adult basic education classes, but went on to employment training classes, apprenticeships, and 
job placement activities. 

As one of very few interventions that sought to facilitate both the parenting and employ- 
ment roles simultaneously, the New Chance Intervention provokes a series of important ques- 
tions. The young mothers who participated in New Chance needed to proceed with lives in which 
they were both parents and sources of economic support to their families. On the one hand, per- 
haps young mothers thrive in a program environment that explicitly recognizes the balancing act 
they need to master in their lives and, accordingly, make progress in both spheres. On the other 
hand, in a population that is at high risk by dint of young age, poverty, and difficult life circum- 
stances, perhaps it is overly ambitious to seek to enhance both roles simultaneously. Does the 
need to incorporate classes and workshops addressing both the maternal and employment roles, 
as well as program components for the children, into the intervention’s programming dilute the 
program’s dosage to an extent that undermines each set of priorities? Were these at-risk mothers 
stressed rather than aided by the comprehensiveness of the program and its expectation of change 
in multiple dimensions of their lives? 

Within this intervention and for this population, then, it is particularly important that we 
ask whether parenting behavior improved and whether other data for the same families indicate 
that it improved along with positive changes in mothers’ education, employment skills, and em- 
ployment. Given the richness of the data available on a range of further possible influences on 
children’s development in this sample, it is also particularly important that we ask about the role 
that parenting behavior played in shaping child outcomes relative to such other factors as chil- 
dren’s experiences in child care, mothers’ psychological well-being, mothers’ welfare receipt, 
educational attainment, acquisition of job skills, employment, and earnings. 

In addition to the substantive issues that can be addressed by embedding observational re- 
search within the larger New Chance Evaluation, doing so also creates an unusual opportunity 
for examining the process of measuring parenting behavior and the measures themselves. Most 
observational studies of parent-child interaction have been carried out with small samples 
(samples of 30 to 50 families are not unusual in such research; see, for example, Denham, Ren- 
wick, and Holt, 1991; Landry et al., 1990). Yet such observational research may be an important 



addition to the set of measures in multisite evaluation research with a focus on parenting. This 
approach may help to diminish both problems that occur when predictors (parenting measures) 
and outcomes (measures of children’s development) come from the same source (maternal re- 
port), 3 and response biases in parenting measures that request information directly from mothers 
or interviewers. This approach may also yield more detailed and qualitative measures of the 
mother-child relationship. 

At the same time, carrying out observational research within the framework of a larger 
evaluation provides an opportunity to examine observational measures of parenting in light of 
extremely rich data from interviews on the mothers and families. It is rare for observational 
studies to include information with the amount of detail available in the present study on such 
issues as the families’ economic circumstances, residential situation, mothers’ education attain- 
ment and literacy, and mothers’ psychological well-being. Because the follow-up surveys of the 
New Chance Evaluation provide interview-based measures of parenting, it is also possible to 
look at the relationship of observation- and interview-based parenting measures. 

There are a variety of ways to implement observational research in a multisite study. One 
way is to develop a collaboration among university research groups and rely upon these research 
groups themselves to collect the data. In such a strategy, professors and their students at the col- 
laborating university sites participate in the development of a common observational study pro- 
tocol and agree upon standardized procedures for collecting and coding the data. Each university 
team implements the protocol within a sample at its site. The videotapes from all sites are then 
coded jointly. This strategy was adopted by the National Institute of Child Health and Human 
Development Study of Early Child Care (Friedman, 1995). A second way to extend the use of 
observational procedures is to rely upon survey interviewers for the collection of observational 
study data. In this strategy, research teams again develop the observational study protocol. How- 
ever, they also adapt the protocol so that it can be administered by survey interviewers. The ob- 
servational study data are then collected by survey interviewers as part of their other data collec- 
tion activities within a sample. This strategy was adopted by the study of parent-child 
interactions embedded within the Teenage Parent Demonstration (Aber, Goodman, and Morris, 
1993; Brooks-Gunn and Berlin, 1993). 

The New Chance Observational Study chose the second strategy, but with a commitment 
from the start to examine the feasibility, problems, and advantages of this strategy. In order to 
carry out this self-reflection, measures have been included in the embedded study to examine the 
issue of mothers’ subjective responses to the completion of the observational procedures in this 
manner. In addition, following completion of the study, we summarized available records on the 
extent to which the survey interviewers adhered to and departed from the carefully scripted ob- 
servational study procedures (see Chapter 12 of this monograph). 

This monograph is organized around the four broad issues that the New Chance Observa- 
tional Study sought to address. Part I provides a detailed examination of program impacts on 
measures of the mother-child relationship (issue 1), contrasts measures of parenting derived from 

3 This problem, correlated error associated with consistent method effects, is discussed in some detail in Chapter 

10 . 
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observational and interview contexts (issue 2), and examines the implications of parenting be- 
havior, relative to other factors, for child outcomes (issue 3). Part II provides an assessment of 
the strategy of implementing an observational study using a survey research approach (issue 4). 

In order to fully understand the goals, procedures, and findings of the New Chance Ob- 
servational Study, it is necessary first to understand the context of this embedded study: the 
population served by the New Chance Program, the intervention strategy itself, the timing and 
content of the full evaluation, and the findings available from the larger evaluation. We include 
in this overview a summary of findings not only from the first follow-up of the New Chance 
Evaluation, but also recently released findings from the final follow-up of the evaluation. Some 
unexpected findings from this final follow-up, particularly concerning children’s development, 
underscore the importance of understanding how the New Chance Program affected parenting 
behavior and how parenting behavior, in turn, contributed to developmental outcomes. 

We turn now to an overview of the New Chance Program and Evaluation, including a 
summary of earlier and recently released findings (see Quint et al., 1994; and Quint, Bos, and 
Polit, 1997). In concluding the introduction, we use the information from this overview to spec- 
ify, in greater detail, the questions that are addressed in the two parts of this monograph. 



II. The New Chance Intervention 

New Chance targeted mothers between ages 16 and 22 who were receiving public assis- 
tance, who gave birth at age 19 or younger, and who had dropped out of high school. For this 
target population, New Chance provided a comprehensive and supportive program that aimed to 
help these mothers complete the General Educational Development (GED) certificate and go on 
to develop employment skills and enter employment. In addition to enhancing the mothers’ eco- 
nomic self-sufficiency, New Chance also supported the mothers’ personal development and 
health through life skills education, family planning and health education classes, personal and 
group counseling, and facilitating their access to health care services. As we have noted, New 
Chance was a two-generation program that sought to enhance the development of both the young 
mothers and their children. While mothers were active in the program, their children were enti- 
tled to child care services. In addition, the program sought to improve access to health care serv- 
ices for the children as well as for the mothers. Of particular importance in the present context, 
mothers in the New Chance Program participated in parenting education classes. 

The New Chance Evaluation, like the intervention itself, focused on two generations 
rather than one. The evaluation considered program impacts on children as well as mothers. It 
also entailed separate consideration of shorter- and longer-term program effects, with follow-up 
interviews carried out both 1 8 and 42 months after mothers enrolled. 

A. Special Characteristics of the New Chance Target Population 

The target population of the New Chance Program is of particular importance from a 
policy perspective. In recent years, adolescent mothers have constituted only about 8 percent of 
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the overall welfare caseload in this country (Granger, 1994; 1995). 4 Yet giving birth as an unwed 
teenager places a mother at high risk of eventual poverty and welfare receipt. Examinations of 
the welfare population in the past decade have revealed that a substantial proportion were teen- 
agers at the time of the birth of their first child. For example, Moore (1990) found that 59 percent 
of women receiving Aid to Families with Dependent Children (AFDC) had been teenagers when 
their first child was bom. Estimates in the past decade indicate that approximately three out of 
four adolescent mothers received welfare within five years of the birth of their first child 
(Adams, 1990). Further, nonwhite women who had not completed high school and who began 
receiving welfare as unmarried mothers were particularly likely to receive welfare benefits for a 
sustained period. Bane and Ellwood (1983) found that the average welfare duration for this group 
was 10 years. Finally, the population of unwed teenage mothers is growing. The number of un- 
married women who had a birth between ages 15 and 19 was 87,100 in 1960. By 1992 this num- 
ber had increased to 353,878 (Brown and Stagner, 1996). 

Thus, it is of national importance to consider effective means for enhancing the long-term 
economic self-sufficiency of those who became unwed mothers as teenagers. Policymakers re- 
peatedly note the importance of this target population. Nevertheless, young mothers who are high 
school dropouts have not in fact been widely served. Few states have focused attention on serv- 
ing adolescent mothers with young children. Further, those that do serve them primarily target 
those who are still in school (Quint et al., 1994). 

The distinction between adolescent mothers who are still in school and those who have 
dropped out is important (Granger, 1994; 1995). In a review of the evidence from programs 
serving poor adolescent mothers, Granger concluded that impacts differ for “students” (those 
who have graduated from high school or are still in school) and for those who have dropped out 
before completing high school. Demonstration programs report some positive impacts for ado- 
lescent mothers who are students, but give little cause for optimism among dropouts. For exam- 
ple, Granger (1995) reports that positive five-year impacts on employment and earnings among 
the adolescent mothers participating in Project Redirection were driven entirely by the subset that 
was still in school at the onset of the program. 

The fact that few intervention efforts have been addressed to a group of such policy im- 
portance is attributable at least in part to the difficulty and complexity of serving unwed teenage 
mothers who have dropped out of school (Granger, 1994; 1995). Quint and colleagues (Quint et 
al., 1994; Quint and Egeland, 1995) point out that the child care needs, limited education, and 
limited history of employment in the group of young high school dropouts make them a group 
with poor prospects of employment as well as costly to serve. Further, data from the New Chance 
Evaluation make it clear that these young mothers face difficult life circumstances that make 
such a target population unusually hard to serve. 

New Chance staff documented extremely high rates of serious problems among the young 
mothers in the program (Quint and Egeland, 1995). Nearly half (48 percent) had such serious 

4 The nation’s largest cash welfare program, Aid to Families with Dependent Children (AFDC), was replaced by 
the Temporary Assistance for Needy Families (TANF) program under legislation signed into law in August 1996. 
The characteristics of welfare recipients under TANF are not yet known. 
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housing problems as being evicted, being asked by the primary tenant to leave, or lacking a per- 
manent residence entirely. Seventeen percent had family members who were involved in sub- 
stance abuse; 12 percent of the mothers themselves used alcohol in a way that interfered with 
program participation and 15 percent used illegal drugs in such a manner. Physical and emotional 
abuse touched the lives of many of the young mothers: 12 percent had experienced physical 
abuse as children and 10 percent sexual abuse. Sixteen percent were reported or observed to be 
battered by a current partner and 6 percent by someone other than the current partner. The young 
women were sometimes discouraged by key individuals in their lives from participating in the 
program: 15 percent by partners and 9 percent by their own mothers. 

Strikingly, almost three-quarters had experienced at least one of these unques- 
tionably serious situations.... The personal problems faced by the large majority of 
New Chance clients makes them an indisputably hard-to-serve group (Quint and 
Egeland, 1995, p. 107). 

While dysfunctional parenting in the family of origin was not documented in a quantita- 
tive manner by New Chance staff, a qualitative study of 50 mothers who had participated in New 
Chance (34 of whom had completed their GEDs and 16 of whom had not) identified this as a key 
issue. Discussing the group that left New Chance without completing the GED in this descriptive 
study, Quint and Musick (1994) note that 

the majority of these young women, while they were growing up and throughout 
their adolescent years, have suffered continuously poor nurturing, repeated sepa- 
rations (or outright abandonment), or loss of mothers or other emotionally signifi- 
cant parental figures. Only five of the 16 young mothers were raised consistently 
by their own mothers. 

The unmet needs for nurturance by some of the young New Chance mothers served to undermine 
their steps toward psychological and economic autonomy and their ability to parent their own 
children. 

In sum, New Chance targeted a critical population, but one burdened by serious problems 
(including for some a lack of supportive parenting in the family of origin), and one for whom 
there has been limited effective programming. The descriptions of parenting in the families of 
origin of some New Chance participants highlight both the importance of addressing parenting 
behavior within this intervention and the possible difficulty of doing so effectively. 

We turn now to a description of the New Chance Program model, how the program was 
intended to unfold, the nature of program services, and the program’s short- and long-term goals. 
In the larger evaluation, Quint and colleagues have distinguished between the program model 
and the actual experience of program participants — for example, documenting discrepancies 
between intended and actual levels of participation (Quint, Fink, and Rowser, 1991; Quint et al., 
1994; Quint, Bos, and Polit, 1997.). In our summary of findings from the larger New Chance 
Evaluation below, we briefly summarize these findings on actual participation for the full 
evaluation sample. In addition, in Chapter 3 we document actual levels of participation in New 
Chance Program components for families in the embedded observational study, focusing espe- 
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daily on those aspects of the program hypothesized to affect parenting. Here, however, our focus 
is on the program “blueprint.” 

B. Program Phases 

In its goal of seeking to improve the employability of the young mothers in New Chance, 
this program looked beyond the completion of the GED to consider the eventual employment of 
program participants. As a result, the program had two phases. Phase 1, lasting several months, 
emphasized completion of the GED. Mothers were expected to attend 20-30 hours per week of 
classes, with mornings (2-3 hours) devoted to adult basic education and GED preparation and 
afternoons to a variety of other classes and workshops (including parenting classes; see below). 
Phase 2 was initiated when mothers had completed the GED or been in the program for five 
months. The goal of this phase was to help mothers enter and keep jobs with benefits and the 
possibility of advancement, through a choice of vocational training, work internships, and further 
education. Throughout both phases, a case manager worked with each mother, monitoring ac- 
tivities and, especially in phase 2, guiding the choice of activities and contacts with other agen- 
cies. The program provided for up to 18 months of program participation, and an additional year 
of follow-up case management. 

C. Program Sites 

New Chance was piloted at 6 sites and actually implemented at 16 sites in 10 states. The 
program was successfully implemented by organizations with various service traditions, includ- 
ing community service organizations, government agencies, adult high schools, and community 
colleges (Quint, Fink, and Rowser, 1991). 

D. Program Services 

Program guidelines specified types and amounts of services to be delivered at each site. 
Program services fell into three broad categories: (1) human capital development, (2) personal 
development of the mothers, and (3) development of the children. 

1. Human Capital Development Services. Services aimed at mothers’ human capital 
development, or skills and knowledge needed for employment, were a primary focus of the New 
Chance Program and were allocated a high proportion of program time and resources. During 
phase 1, mothers participated in classes aimed at successful completion of the GED. Mothers 
also participated in classes aimed at employment and career awareness, which focused on a vari- 
ety of occupations and the skills needed to enter them. In phase 2, mothers participated in occu- 
pational skills training tailored especially to jobs in the local market, full or part-time work in- 
ternships, job placement activities, and further education at community colleges. 

2. Personal Development Services. Personal development classes and workshops 
aimed to address the many problems that the young women faced in their everyday lives. Life 
skills training focused on mothers’ communication and decision-making skills in sexual contacts, 
relationships, parenting situations, and work. Health education classes and health services in- 
cluded components on nutrition, stress management, physical and emotional abuse, substance 
abuse, childhood immunization, sexually transmitted diseases, and depression and anxiety. Fam- 
ily planning issues, addressed in both group sessions and individual counseling, aimed at encour- 
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aging the young women to delay subsequent pregnancy and childbirth. Mothers also received 
individual counseling through meetings with their case managers. The young mothers met indi- 
vidually with their case managers once every two weeks during phase 1 of the program and gen- 
erally less often during phase 2. Together, clients and case managers developed an individual 
program plan and reassessed progress periodically. Program guidelines recommended caseloads 
of no more than 25 clients (15 clients, if case managers had other responsibilities). 

3. Child Development Services. Children were provided access to pediatric health 
services on an as-needed basis, mostly at off-site clinics and hospitals. New Chance offered child 
care at no cost to all participants for as long as they remained active in the program. In 12 of the 
16 program sites such care was offered on site, although for two of these, only drop-in care was 
available. Sites that arranged for off-site care for clients’ children also monitored the quality of 
care provided. Guidelines for on-site and off-site child care called for programs that were sup- 
portive to young children’s development. 

Of particular importance in the present context, mothers in New Chance participated in 
parenting education classes, which involved a balance of (1) presentations aimed at covering 
specific content and (2) group discussions directed by the concerns of participants rather than a 
specific curriculum. Program guidelines called for coverage of such issues as stages of develop- 
ment (physical, emotional, and cognitive), developmentally appropriate cognitive stimulation, 
strategies for setting reasonable behavioral limits at different ages, and building self-esteem and 
trust in children. Guidelines also called for joint mother-child sessions once a month to provide a 
context in which staff could demonstrate activities and interactions, observe mothers interacting 
with their children, and provide feedback. 

E. Short-Term and Longer-Term Program Goals 

New Chance had both short-term and longer-term program goals. In the short term, the 
program components summarized above aimed to enhance maternal educational attainment, 
health status, emotional well-being, parenting behavior, and the mother-child relationship; also, 
to help mothers postpone further pregnancy or childbearing and to expose children to high- 
quality child care. Longer-term goals included increasing mothers’ employment and earnings, 
reducing welfare receipt, and improving children’s development across the domains of health, 
cognitive development, and social development. 



III. The New Chance Evaluation 

A. The Evaluation Design 

A rigorous evaluation of the New Chance Program has been conducted by the Manpower 
Demonstration Research Corporation (MDRC), consisting of three key components: an imple- 
mentation evaluation, an impact evaluation (that is, an examination of differences between ex- 
perimental and control group families on outcome variables), and a cost-benefit evaluation. It is 
crucial to note that the impact analysis, unlike many other program evaluations focusing on low- 
income families, encompasses measures of the well-being of both the mothers and their young 
children. Information was collected from the mothers just prior to their assignment to an experi- 
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mental or control group (at “baseline”) and through in-home interviews at 18 and 42 months after 
enrollment in the evaluation. Findings from the implementation study (Quint, Fink, and Rowser, 
1991) and the 18-month survey (Quint et al., 1994) have already been published. Findings from 
the 42-month survey, including data regarding children’s cognitive development, socioemotional 
development, and health, have just been released (Quint, Bos, and Polit, 1997). 

Each of the program sites was asked to recruit approximately 150 participants for the 
evaluation. Those mothers who volunteered for the program, met the eligibility criteria, 5 and 
agreed to participate in the evaluation were randomly assigned to an experimental or control 
group, with approximately two-thirds of those enrolled in the evaluation being assigned to the 
experimental group and one-third to the control group. (A two-to-one random assignment ratio 
was adopted because it was reasoned that sites would respond more favorably to random assign- 
ment, and probably would be able to recruit applicants more readily if young women knew that 
their chance of admission to the program was better than 50-50. This ratio also facilitated the ex- 
amination of findings for key subgroups within the experimental group.) 

The evaluation sample was enrolled in the study between August 1989 and July 1991. 
Altogether 2,322 mothers were enrolled. When a mother had more than one child, one was ran- 
domly chosen prior to the 18-month follow-up as the “focal child,” or child to be focused upon in 
consideration of program impacts on children. Experimental group mothers were enrolled in the 
New Chance Program. Control group mothers were not allowed to participate in New Chance, 
but were free to seek other services in the community. Impact analyses in the New Chance 
Evaluation thus consider not how New Chance affected mothers and children compared with 
families receiving no relevant services, but rather how New Chance affected families above and 
beyond the services that the control group families sought and received on their own. 

Of the 2,322 mothers enrolled in the evaluation sample, 2,106, or 90.7 percent, were in- 
terviewed in their homes approximately 18 months after enrollment (the first follow-up). For the 
second follow-up, carried out approximately 42 months after enrollment, 2,105, or 90.6 percent, 
of the mothers initially enrolled were interviewed in their homes. These response rates are ex- 
tremely high for a sample of high-risk families. 6 

In addition to the measures designed to capture program impacts for mothers, the final 
follow-up of the New Chance Evaluation provides a number of measures of the children’s devel- 
opment. These outcomes, measured approximately 42 months after mothers enrolled in the dem- 
onstration as part of the final follow-up, permit consideration of program impacts on the chil- 
dren. 7 During the 42-month interview, a direct assessment of the children’s cognitive 



Eligibility for the program was determined through an initial interview, during which data on the young 
women’s characteristics at baseline were recorded on the New Chance Enrollment Form. A reading test (the reading 
part of the Tests of Adult Basic Education, TABE) was also administered to program applicants at baseline (see 
Quint et al., 1994, p. 25, for further details). 

6 Data were sufficiently complete to permit analyses for 2,088 of the 2,106 interviews completed at the 18- 
month follow-up and for 2,079 of the 2,105 interviews completed at the 42-month follow-up. 

7 See Polit (1996a) for a detailed description of each measure, reasons for selection of the measure within the 
New Chance Evaluation, and reliability and validity data. 
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development, the School Readiness Component of the Bracken Basic Concept Scale (Bracken, 
1984) was administered. This scale is a measure of receptive language with content reflecting 
school readiness. The subtests that constitute the School Readiness Component of the scale as- 
sess the specific concept areas of colors, letter identification, numbers/counting, comparisons, 
and shapes. For the New Chance Evaluation sample, internal consistency reliability for this 
measure is quite high at .98. As an indication of the validity of the measure, for those children in 
the New Chance Evaluation sample whose teachers completed the Teacher Questionnaire, the 
total score on the School Readiness Component varied significantly by teacher rating of chil- 
dren’s academic performance (Polit, 1996b). 

To assess child outcomes in the area of socioemotional development, two measures were 
obtained from mothers: the Behavior Problems Index (Zill, 1985) and the Positive Behavior In- 
dex (Polit, 1996a). These two measures were also obtained through a survey of the teachers for 
those focal children in the sample already attending school, Head Start, or a structured, academi- 
cally oriented preschool program. Teacher survey data are available for 835 or 79.5 percent of 
these children (86.2 percent of the 969 children whose mothers allowed their children’s teachers 
be contacted). The Behavior Problems Index asks an adult who is familiar with a particular child 
to report on the incidence of problems in six areas: antisocial, anxious/depressed, headstrong, 
hyperactive, dependent, and peer conflict/withdrawn behavior. The index yields a total score as 
well as subscale scores for each of the six behavior areas. This measure was chosen for the New 
Chance Evaluation because it is widely used (for example, in the Child Health Supplement of the 
National Health Interview Survey and the Child Supplement of the National Longitudinal Survey 
of Youth) and has been shown to have strong reliability and validity in previous work. Within the 
New Chance Evaluation sample, children whose mothers reported a discipline problem that re- 
sulted in a communication from school had higher average total Behavior Problems Index scores. 
Scores on the index were also related to interviewer ratings of the children’s behavior during the 
42-month follow-up (Polit, 1996a). 

The Positive Behavior Index was developed specifically for the New Chance Evaluation 
when a review of existing measures found none that both addressed positive social behavior in 
children and could be considered appropriate for administration in a disadvantaged sample (Polit, 
1996a). The Positive Behavior Index provides a total score and scores for three subscales: com- 
pliance, social competence, and autonomy. Internal consistency reliability coefficients for this 
newly developed measure ranged from .77 to .94 for maternal report. The total score on the 
Positive Behavior Index was again related to the report of a communication from school regard- 
ing a discipline problem and to the interviewer’s ratings of the child’s behavior. 

In addition to the Behavior Problems Index and the Positive Behavior Index, mothers and 
teachers provided a series of further ratings. Mothers rated children’s academic performance, 
overall health and use of health services. Teachers provided ratings on children’s general aca- 
demic performance and ability, self-esteem, motivation, overall school adjustment, and how well 
they got along with teachers and students. Teachers also reported on disciplinary action taken 
and whether they had had meetings with mothers for a general discussion of their children. Many 
of the items from the New Chance Teacher Questionnaire were taken or adapted from the 
Teacher Questionnaire used as part of the 1981 National Survey of Children (Polit, 1996b). 
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B. Key Findings from the Implementation Study 



MDRC’s report on the implementation of New Chance (Quint, Fink, and Rowser, 1991) 
notes that despite their varying service traditions, the 1 6 New Chance sites successfully mounted 
all phase 1 activities. The sites found education among the easiest of the phase 1 services to put 
in place, and employability development and individualized family planning services the most 
difficult. Phase 2 services, because they were offered off site and through a variety of outside 
services, were harder to implement in a uniform manner. The outside agencies did not have the 
same familiarity and skill in working with the population targeted by New Chance. Further, New 
Chance case managers found it difficult to continue their biweekly contacts with their off-site 
clients given their on-site caseloads. 

Analyses indicate discrepancies between intended and actual levels of program participa- 
tion. Absenteeism was a problem in the New Chance Program, with approximately half of those 
enrolled actually attending program activities on a typical day. While the young mothers in the 
experimental group clearly received more services than those in the control group, control group 
mothers sought and received non-New Chance services in the community on their own initiative; 
for example, as reported in the New Chance 18-month follow-up, while 85 percent of experi- 
mental group mothers participated in some education classes, 60 percent of control group moth- 
ers did as well. The discrepancy in participation was greater for program components other than 
education; for example, approximately three times as many mothers in the experimental group as 
in the control group participated in parenting education classes. These findings are in keeping 
with the fact that mothers who volunteered for New Chance were primarily seeking education 
rather than personal development services. 

Table 1.1 illustrates the discrepancy between intended and actual program participation. 
This table compares mean hours of actual program participation with hours of program partici- 
pation specified in program guidelines for experimental group mothers over the first 18 months 
of program participation. As can be seen, while program guidelines called for between 32 and 64 
hours of participation in parenting classes, on average this sample of mothers participated for less 
than 18 hours. The discrepancy between intended and actual hours of program participation does 
not reflect a problem with service delivery; all sites provided services to the extent specified. 
Rather, the discrepancy reflects attendance issues. Perhaps in keeping with the many difficulties 
in their lives, and the voluntary, rather than mandatory, nature of the program, New Chance 
mothers showed inconsistent attendance. 

Differences in levels of program participation were, not unexpectedly, linked to back- 
ground characteristics of the mothers. Experimental group mothers who participated more in 
New Chance Program activities had more positive educational histories (higher educational at- 
tainment, less likelihood of having repeated a grade), were more likely to have parents who had 
graduated from high school or completed a GED, were more likely to be using birth control and 
to have received medical care on program entry, and reported more emotional support. Among 
racial/ethnic subgroups in the evaluation, Hispanic mothers attended New Chance activities 
most, followed by black mothers and then white mothers. 

Analyses of program costs at 18 months indicate that on average the program cost the 
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Table 1.1 



Program Guidelines for Participation and Mean Hours of Actual Participation 
in New Chance Within 18-Month Follow-Up Period 



Program Component 


Number of Hours 
Specified by Program 
Guidelines 


Mean Hours of 
Participation in 
Component 


Education 


192 


101.0 


Employability development 


48 


26.7 


Family planning 


6 


6.3 


Health education 


24 


10.5 


Parenting education 


32-64 


17.6 


Life skills 


36-39 


20.5 


Other group activities (e.g., 
field trips or social events) 


— 


19.3 


Average hours of participation in all 
components 3 


— 


297.6 


Sample size 




1,408 



Sources: New Chance Management Information System (MIS) data. 

Program guidelines from MDRC 1991 implementation report. 

Notes: Calculations for this table used data for 1,408 experimentals for whom there were 

18-month follow-up survey data, including values of zero for those who were randomly 
assigned to New Chance but did not participate. 

Dashes indicate unavailable data. 

a Excludes individual counseling and college classes. 
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sponsoring agencies $5,073 per mother in the experimental group, excluding the cost of child 
care. Child care cost an additional $2,573 on average, and costs from other agencies totaled an 
average of $1,380. 

C. Characteristics of the Evaluation Sample at Baseline 

Examination of the characteristics of the evaluation sample 8 at the time they enrolled in 
the evaluation confirms that this was indeed a group of disadvantaged young mothers with lim- 
ited education. This group of mothers averaged just under 19 years of age at the time they en- 
rolled in the demonstration. On average, they had given birth to their first child before age 17. 
More than three-fourths of the sample (77 percent) were from minority racial/ethnic groups, and 
fewer than 1 in 10 had ever been married. About one-third of the mothers already had two chil- 
dren or more, and a majority had a child under age 2. A little over one-third of the mothers had 
dropped out of school before their first pregnancy. Mothers averaged more than two years out of 
school and had reading skills just above the 8th grade level. Few mothers had worked full time 
for a sustained period. Nearly two-thirds of them (64 percent) had grown up in a family with 
some welfare receipt. Ninety-five percent of the mothers were currently receiving AFDC, nearly 
all as heads of their own households. A much higher proportion of mothers in this sample than in 
broad community-based samples (53 percent), received scores on a depression index indicating 
that they were at risk for clinical depression. 

D. Key Findings from the Impact Evaluation: The 18- and 42-Month Follow-Ups 9 

1* Education and Literacy. Analyses of 18-month program impacts — that is, dif- 
ferences between experimental and control group mothers across particular outcomes at this time 
— indicate that some of the articulated short-term program goals but not others were effectively 
addressed. A higher proportion of mothers in the experimental group than in the control group 
(37 percent as opposed to 21 percent) had completed the GED, and more mothers in the experi- 
mental group had some college credits by the time of the 18-month follow-up. Yet no group dif- 
ferences were found in mothers’ literacy levels, with about 40 percent of each group reading at or 
below the 7th grade level. Mothers averaged a 7.6 grade reading level. 

At the 42 -month follow-up, several markers of educational attainment continued to point 
to positive program impacts. The proportion of mothers who had completed the GED grew for 
both the experimental and control groups between the 18- and 42-month follow-ups, but the pro- 
portion of mothers with a GED continued to be significantly higher in the experimental group 
(45 percent versus 33 percent). A significantly higher proportion of mothers in the experimental 



8 The figures for baseline characteristics of the sample in this paragraph are based on data for the 2,088 respon- 
dents included in analyses of the 18-month survey as reported in Quint et al., 1994. For baseline characteristics of 
the full New Chance sample of 2,322, see Quint et al., 1994, Appendix B. In that report, statistical tests showed no 
large or systematic differences between the two samples. 

9 In the following section, 18-month findings are from the New Chance interim report (Quint et al., 1994). 
Forty-two-month findings are from the final report (Quint, Bos, and Polit, 1997). The final report also includes se- 
lected 18-month outcomes analyzed for the sample of families included in the 42-month report. As this was a 
slightly different sample from that used in the analyses for the interim report, the numbers reported here from the 
1 8-month interim report differ slightly from the selected 1 8-month findings presented in the 42-month report. 
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group had either a GED or a high school diploma (52 percent versus 44 percent) and, further, a 
higher proportion of mothers in the experimental group had completed some college credits (14 
percent versus 11 percent). 10 At the same time, a smaller proportion of mothers in the New 
Chance experimental group (7 percent) than in the control group (10 percent) had obtained a high 
school diploma, perhaps because of the program’s emphasis on completing the GED. 

2. Markers of Economic Self-Sufficiency. Findings on economic self-sufficiency at 
the 18-month follow-up for the New Chance sample suggest an “opportunity cost” of participat- 
ing in educational activities. That is, in the short run, participation appeared to be associated with 
lower rates of employment and lower earnings, as mothers initially focused on education rather 
than employment. At 18 months, control group mothers had higher cumulative earnings. More 
control group mothers had been employed in the first months after random assignment. Across 
the 18-month follow-up period, 43 percent of mothers in the experimental group and 45 percent 
of those in the control group had worked at some point. Over 80 percent of mothers in both 
groups were still on welfare at 18 months. Experimental group mothers were more likely to be 
living in public housing or receiving rent assistance. 

Although earnings and labor market experience are forgone during periods of investment 
in education and job skills training, the expectation is that this initial cost will eventually yield 
higher rates of employment and better earnings. Yet at the time of the 42-month follow-up, no 
group differences were found for mothers’ employment or earnings. Experimental and control 
groups did not differ significantly in terms of the proportion of mothers employed at all or em- 
ployed full time. Further, no significant differences were found in earnings or hourly wages. 
Without a further follow-up for the New Chance Intervention planned, it is impossible to say 
whether this pattern will hold or whether the employment patterns will diverge again in the fu- 
ture, with mothers in the experimental group eventually benefiting from their increased educa- 
tional attainment. 

3. Fertility . At the time of the 18-month follow-up, there was no indication that ex- 
perimental group mothers had successfully delayed subsequent childbearing. More than one 
quarter of mothers in both groups had had a subsequent child at this follow-up point. Although 
there were no group differences in terms of childbirth at the time of the 18-month follow-up, ex- 
perimental group mothers actually had had a significantly higher rate of pregnancy and abortion 
than control group mothers. 

By the time of the 42-month follow-up, there were no indications of important differences 
in the fertility behaviors of mothers in the experimental and control groups. The proportion of 
mothers who had had a pregnancy or birth did not differ significantly by group, nor did the aver- 
age number of post-enrollment pregnancies. At the 42-month follow-up, mothers in the experi- 
mental and control groups reported similar patterns of sexual activity and contraceptive use (for 
example, the proportion of mothers reporting that they were sexually abstinent and not pregnant, 
or sexually active and contracepting regularly, did not differ across groups). There is no evi- 
dence, then, that the New Chance Program had impacts on mothers’ fertility behaviors. 



l0 This difference, while numerically small, was statistically significant. 
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4. Residence . During the first 18 months of the evaluation, mothers in the experi- 
mental group showed more shifts in residence than mothers in the control group. At the time of 
the 18-month follow-up, mothers in the experimental group were more likely to be living with a 
partner or husband and less likely to be living with a parent or grandparent. Residence and fertil- 
ity patterns were linked. A higher rate of subsequent pregnancy among experimentals occurred 
only for those living with a husband or partner. 

Findings from the 42-month follow-up provide further indications of a group difference 
in residence patterns. The average number of times that mothers had moved (from the time of the 
child’s birth to the time of the 42-month follow-up) was significantly higher in the experimental 
group (averages of 4.0 and 3.7 times in the experimental and control groups, respectively). Fur- 
ther, more mothers in the experimental group reported having had trouble finding a good place to 
live in the past year (41.9 and 37.5 percent in the experimental and control groups, respectively), 
and more experimental group mothers were living without any of their children at the 42-month 
follow-up (4.9 percent as opposed to 2.8 percent). At the second follow-up there were no longer 
significant differences in the proportion of mothers living with a husband or partner or living 
with a parent or grandparent. Yet a small but significantly higher proportion of mothers in the 
experimental group were living in “other” arrangements at 42 months (living with friends, alone, 
or in an institution). 

5. Maternal Psychological Well-Being . Findings regarding maternal psychological 
well-being differed substantially for the first and second follow-up points. At 18 months, no pro- 
gram impacts were detected regarding maternal depression, stress, sense of control over events, 
or health. However, mothers in the two groups differed in their reports on two aspects of social 
support. Experimental group mothers were less likely than control group mothers to say that they 
had no one to turn to for emotional support; these mothers also indicated that they were more 
satisfied with the social support that they were receiving. 

By 42 months, however, there were indications that New Chance had had negative im- 
pacts on mothers’ psychological well-being. Average scores on the measure of depression were 
significantly higher in the experimental group at this later time. While depression scores declined 
in both the experimental and control groups over time from baseline to 42 months, the decline 
was significantly greater for the control group mothers. A higher proportion of mothers in the 
experimental group reported that they felt stressed much or all of the time in the past month, and 
their satisfaction with standard of living was significantly lower. At the same time, there were no 
indications at the final follow-up point that mothers in the experimental group felt a greater sense 
of social support. 

Several possible bases of these unexpected findings on maternal subjective well-being are 
discussed in detail in the full report of 42-month findings (Quint, Bos, and Polit, 1997). One pos- 
sibility is that New Chance raised expectations for improvements in life circumstances, but real- 
izing these improvements proved much more difficult than expected. As a result, New Chance 
mothers may have experienced “dashed hopes,” manifested in greater stress, depression, and dis- 
satisfaction with economic circumstances. 
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6. Parenting Behavior As Documented in the Context of Interviews . At 18 

months, statistically significant differences were found on two of the interview-based measures 
of parenting. On a measure of control/punitiveness, experimental group mothers reported less 
harsh and more authoritative (that is, supportive but firm) disciplinary attitudes. Further, on the 
component of the Home Observation for Measurement of the Environment (HOME) Inventory 
(short form) that measures mothers’ provision of emotional support to the focal child, experi- 
mental group mothers were found to be providing more emotional support. 

At 42 months, no overall group differences were found on the subscales of the HOME 
Inventory. A positive program impact on the HOME total score was found, however, for the sub- 
group of mothers who did not show indications of depression when they entered the program. In 
keeping with the greater indications of stress and depression among experimental group mothers, 
mothers’ report of stress in parenting was higher in the experimental group at 42 months (as 
measured by both a total parenting stress score and a subscale of aggravation with child rearing). 

7. Child Care . By the 18-month follow-up, children in the experimental group were 
substantially more likely to have experienced nonmatemal care, especially center-based child 
care. Experimental group children were also more likely to have been in a regular child care ar- 
rangement before age 1 . This pattern reflects, at least in part, the fact that many New Chance 
sites offered on-site center day care. 

By the 42-month follow-up, the two groups had converged in terms of reliance on child 
care. No group differences were found in terms of current use of a child care arrangement, or 
participation specifically in center child care, family day care, regular care by a grandparent, 
Head Start, or school. While a higher proportion of children in the experimental group had par- 
ticipated in a day care center or preschool over the full course of the evaluation, this difference 
was largely attributable to the earlier months of program participation. 

8. Child Outcomes , Child outcomes are available from the 42-month follow-up and 
the teacher survey. Again unexpectedly, negative program impacts were detected across several 
measures. Mothers’ reports of children’s behavior problems were significantly higher in the ex- 
perimental group (overall and on three of six Behavior Problems Index subscales). At the same 
time, mothers’ reports of children’s positive behaviors were significantly lower in the experi- 
mental group (overall and for all three Positive Behavior Index subscales). There were no group 
differences in mothers’ overall ratings of their children’s health, in the proportion of children 
who had been hospitalized at least once, or in the proportion of children who had had an emo- 
tional trauma. Yet mothers in the experimental group reported that a higher proportion of their 
children had had an injury, poisoning, or accident requiring medical attention. 

As we have noted, for some of the children in the evaluation sample, ratings of behavior 
problems and strengths were available for the same children from both the mother and a teacher. 
Within this smaller sample of children already attending a school or academically oriented pre- 
school, significant differences between experimental and control groups emerged more often in 
maternal reports regarding the child than in teacher reports. Specifically, mothers in the experi- 
mental group rated the academic performance of their children less favorably than mothers in the 
control group. In addition, more mothers in the experimental group reported being notified by the 
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school of behavior problems. By contrast, teachers rated the academic performance of the chil- 
dren in the two groups similarly. Teachers rated daughters of mothers in the experimental group 
as presenting more behavior management problems requiring parental notification, but saw no 
group difference in the behavior of sons. The teacher survey found no group differences on a se- 
ries of measures that only the teachers reported: teachers saw no significant difference in the pro- 
portion of children in the experimental and control groups who had problems getting along with 
students or teachers, with self-esteem or motivation, or in terms of their overall adjustment. 

The fact that indications of negative program impacts on children’s behavior were seen 
more through the eyes of the mothers than the teachers deserves further scrutiny. Several inter- 
pretations are possible for this discrepancy in child outcome findings by source. One possible 
explanation focuses on context. Children manifest different behaviors at home and at school. 
Perhaps differences in the behavior of children in the experimental and control groups surface 
more often in the context of the home environment. Such an interpretation seems plausible par- 
ticularly if children are more restrained in the context of the greater structure and behavioral ex- 
pectations of school, or if problem behaviors that are reported more often in the experimental 
group are directed specifically toward the mother or other family members. 

A second interpretation focuses on the informant : mothers’ perceptions of their children’s 
behaviors may have been affected by participation in New Chance, whether or not the children’s 
behavior actually differed. As we noted earlier, New Chance mothers exhibited higher levels of 
depression than control group mothers at 42 months, which might have colored their perceptions 
of their children. Another possibility concerns day care participation. Because more children of 
mothers in the New Chance experimental group than in the control group had been in day care, 
mothers in the experimental group had had, over time, more contact with child care providers. It 
is possible that conversations with child care providers alerted mothers to problems regarding 
their children’s social behavior that child care providers thought needed to be addressed within a 
classroom context. Such problems might have been occurring with equal frequency among chil- 
dren who had not attended day care (and with equal frequency among children of mothers in the 
experimental and control groups). Yet with more frequent contact with child care providers, 
mothers in the experimental group may have been alerted to such problems with greater fre- 
quency. When asked about their children’s behavior, they may have therefore been more likely to 
acknowledge the presence of behavior problems. Thus, participation in New Chance might have 
altered the perception of behavior if not actual child behavior. 

A third interpretation focuses on sample. The sample for the teacher survey differed from 
the full evaluation sample in that children were somewhat older. In addition, all of the children in 
the teacher survey sample were currently participating in some form of academically oriented 
school or preschool. Children in academically oriented settings and their families may have dif- 
ferent characteristics from those not making use of such programs (Zaslow et al., 1997). Perhaps 
the characteristics that predispose some families to the use of educational settings also buffer 
against negative program impacts on child outcomes as they would be manifested in school set- 
tings and reported on by teachers. 
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E. Summary of Findings 



These complex and sobering findings from the full New Chance Evaluation speak to the 
difficulty of working with the group of very young welfare mothers who have dropped out of 
school. Despite the many obstacles facing this group of mothers, the New Chance Program was 
able to bring about positive changes in educational attainment. Yet 42 months after the start of 
the program there was no indication that the enhanced educational attainment had resulted in an 
improvement in mothers’ economic self-sufficiency. The program was also unable to alter moth- 
ers’ fertility behavior. Further, there are unexpected findings that New Chance had negative im- 
plications for mothers’ psychological well-being and residential stability. 

In the present context, the findings on parenting behavior and child outcomes from the 
larger study are particularly salient and they present a paradox. On the one hand, there are mod- 
est indications that parenting behavior was positively affected by the New Chance Program. 
Positive impacts were detected at the first follow-up on two interview-based measures of the 
emotional quality of the mother-child relationship. Yet despite these indications of changes for 
the better in parenting behavior, there were negative program impacts on several important child 
outcomes. Mothers of children in the experimental group perceived their children as having more 
behavior problems and as showing fewer positive social behaviors. Mothers of school-age chil- 
dren in the experimental group described their children as doing less well in school both aca- 
demically and behaviorally. 

A substantial body of research documents that parenting behavior is of great importance 
in shaping child outcomes (reviewed in Maccoby and Martin, 1983; Bomstein, 1995). Why 
would there be negative impacts on child outcomes when New Chance seems to have contributed 
to a positive impact, albeit small, on parenting behavior as measured by interview-based meas- 
ures of parenting? Several explanations seem plausible. First, perhaps the positive program im- 
pacts that the program had on parenting at the first follow-up were of too small a magnitude to 
have influenced subsequent child outcomes in a substantial and positive manner. Second, perhaps 
other aspects of the families’ lives (such as maternal stress and depression or economic depriva- 
tion) were important in shaping child outcomes and overwhelmed any positive implications that 
the program may have had for parenting. Third, perhaps we should scrutinize and question the 
parenting measures collected as part of the two follow-up interviews. Measures of parenting that 
rely on the reports of mothers and ratings of interviewers may be subject to a variety of response 
biases. For example, the fact that these informants were aware of the assignment of families to 
the experimental group versus the control group may have affected the ratings they gave. Perhaps 
a markedly different picture of program impacts would emerge (such as an absence of group dif- 
ferences or a negative program impact on parenting) if measures of parenting behavior were 
based on the ratings of outside observers, especially those unaware of whether families had or 
had not participated in New Chance. 

The embedded observational study makes it possible to weigh these various possibilities 
for a subset of the full evaluation sample: that program impacts on parenting were not of suffi- 
cient magnitude to contribute positively to child outcomes, that other factors overwhelmed a 
positive influence of parenting on child outcomes, and that more rigorous parenting measures 
would yield a different picture of program impacts on parenting. The observational study pro- 
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vides ratings of both the affective and cognitive aspects of mother-child interaction, completed 
by observers less vulnerable to a number of possible response biases, under conditions requiring 
intensive training in the constructs and measures of interest, and high interrater reliability. We 
are thus able in this monograph to extend the examination of program impacts on parenting be- 
havior to a broader set of parenting variables, documented in previous work to be highly sensi- 
tive to family context and predictive of child outcomes, and with fewer concerns about response 
bias. In particular, we ask in subsequent chapters whether the pattern of modest program impacts 
on parenting is corroborated with the observational measures and whether the highly sensitive 
observational measures tell us anything more about the particular kinds of behaviors affected by 
the program. With access to measures of parenting derived from both interview and observa- 
tional settings, the current monograph also examines in detail the issue of whether and how par- 
enting behavior measured in different ways contributes to child outcomes. Finally, we access the 
broad set of variables available from the two follow-up interviews in the full evaluation to ask 
how other factors combine with parenting variables in shaping the child outcomes. 



IV. Questions Addressed by the New Chance Ohservational Study 

A. Questions Concerning Program Impacts on Parenting Behavior and the Role of 
Parenting Behavior in Shaping Child Outcomes 

In order to examine the implications of the New Chance Program for parenting behavior, 
and the role of parenting behavior in turn in shaping child outcomes, the monograph addresses 
the following sequence of questions: 

• How can we describe the parenting behavior of the mothers in the New 
Chance Observational Study overall? Using methods and procedures that 
have been employed in other studies of at-risk populations, what can be said 
broadly about the affective quality and stimulation for early literacy in 
mother-child interactions? 

• Which subgroups appear to be at greatest risk in terms of their parenting 
behavior, and which are providing the most stimulating and supportive 
interactions? Although the mothers in the New Chance Observational Study 
are uniformly young, poor, and of limited education, is there nevertheless 
meaningful variation within this sample in terms of parenting behavior? Do 
the affective quality and literacy-stimulating aspects of interaction vary ac- 
cording to background characteristics such as whether or not the mother re- 
ceived welfare as a child, the age at which the mother gave birth to her first 
child, and the years of schooling she has completed? If there are groups that 
appear particularly strong or particularly at risk in terms of parenting behavior, 
are the patterns the same for the literacy-related and affective dimension of 
interaction? 

® Did the New Chance Intervention have positive impacts on the observa- 
tional measures of parenting behavior? If so, did it have positive impacts 
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across the two domains of parenting behavior examined through direct obser- 
vation of mother-child interaction — the affective quality of interactions and 
the aspects of interaction related to the development of literacy — or were im- 
pacts restricted to one of these key domains? Within each domain, are effects 
detected across multiple parent and child behaviors, or are effects more cir- 
cumscribed? Are findings for the observational measures concordant with the 
program impact results for the interview-based measures of parenting within 
the observational study sample? 

• If positive impacts are detected, did they occur consistently across fami- 
lies with varying characteristics, or were they found primarily for defin- 
able subgroups? For example, are positive impacts documented for those 
mothers who could be said to be at less risk at the start of the evaluation (for 
example, mothers with only one child, with more years of education and less 
indication of school failure) or also for those at greater risk? 

• How do positive impacts come about? If New Chance has positive impacts 
on parenting behavior, can we trace such impacts to the parenting education 
component of the intervention, or are other program components also contrib- 
uting? Are there different explanatory processes for different types of parent- 
ing measures, or do all of the measures follow the same pattern? 

• How do the observational and interview-based measures of parenting re- 
late? What is the degree of overlap among these measures? What distinctive 
information, if any, is provided by the observational measures? When we add 
observational measures of parenting to an evaluation, do we improve our abil- 
ity to predict child outcomes? 

• How important are measures of parenting, relative to other sets of vari- 
ables, in predicting the cognitive and social development of the young 
children in the sample? How do such further variables as maternal educa- 
tional attainment and employment status, maternal depression, current resi- 
dence pattern, and child care history combine with the parenting variables to 
influence child outcomes? 

B. Questions Concerning the Methodology Used to Document Parenting Behavior 

We have noted that the New Chance Observational Study followed a relatively new 
fielding strategy in order to introduce the intensive procedures of observational research into a 
multisite evaluation. The present study reflects the intersection of two traditions: the attempts of 
developmental psychology to extend observational research to larger and more representative 
samples and of survey research to go beyond interview formats to other data collection strategies. 
Our methodology can be seen as a “hybrid” approach that seeks to use the strengths of the survey 
model in tracking and maintaining contact with a population that is difficult to locate and include 
in research, and the strengths of developmental psychology in developing tasks and coding pro- 
cedures for recording interactive behaviors that are of central importance to children’s develop- 
ment. 
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The New Chance Observational Study not only attempted this new approach, but also in- 
corporated tools to evaluate the effectiveness of the strategy. We have mothers’ reactions to the 
observational session (for example, ratings of such items as “the interviewer helped me feel com- 
fortable during the session” and “while I was working with my child on the book and games, I 
felt nervous about how I was doing”), interviewers’ reports of the context for the observational 
session (for example, noise level, interruptions from other children), and coders’ comments about 
the extent to which the script and procedures for the observational sessions were adhered to or 
violated during each session. 

In the second part of this monograph, we critically examine our methodology of carrying 
out observational procedures within a survey research framework, asking specifically: 

• What are the implications of conducting observational research through 
a survey research model? As developmentalists turn to contracted survey re- 
search in order to leverage larger and/or more representative samples, they 
will need to understand the culture of survey research and how this may differ 
from laboratory-based research in order to derive the greatest value from the 
collaboration. 

® How can the body of methodological work on survey measurement in- 
form the study of parenting behavior using a combination of interviews, 
interviewer reports, and observational procedures? While turning to the 
survey community for its data collection capabilities and enhanced samples, 
developmentalists may also wish to draw on the survey field’s corpus of 
methodological work on survey measurement. While most of this work fo- 
cuses on respondents’ answering questions from interviewers, there are im- 
portant implications for observational research as well. 

• How were the laboratory-based observational protocols adapted for sur- 
vey administration, and how was the observational study implemented in 
the field? What challenges were encountered in conducting the observa- 
tional study in this way? The lay survey interviewers who collected the data 
lacked substantive background and familiarity with previous research using 
the observational protocols. They needed a highly structured set of data col- 
lection materials, including a “script” resembling a survey questionnaire, in 
order to present a standardized stimulus situation. This work presented logisti- 
cal challenges in carrying out the field work (for example, coordinating the 
work of two-person teams), situational problems (for example, interruptions 
from other children needing attention, lack of work space), and management 
difficulties associated with the poor “fit” of this type of study with ongoing 
survey operations. 

® How well did lay survey interviewers meet the challenges of administer- 
ing observational research? The observational study required survey inter- 
viewers to extend their repertoires significantly. What have we learned about 
the ability of interviewers to handle such a task and about their potential for 
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such work in the future, when the lessons from the current effort can be incor- 
porated? 

• How did the mothers experience the observational session? What were 
the mediators of participants’ subjective experience of the session? The 

tasks were designed to be sufficiently challenging for the children to need 
their mothers’ help, but not to be unpleasant or overly stressful. How success- 
ful were we at creating this balance? A number of variables could contribute 
to mothers’ sense of stress in the situation or to having had a positive experi- 
ence, such as the interviewer’s skill at administering the protocol, the inter- 
viewer’s experience in interviewing disadvantaged young women, and the age 
of the child. Which, if any, of these are important? 

• What recommendations can we make for future research involving ob- 
taining observational measures of parenting within a survey research 
context? 



V. The Organization of the Monograph 

Part I of this monograph examines the first set of questions concerning program impacts 
on parenting behavior, the relation between observation- and interview-based measures of par- 
enting, and the role of parenting behavior in shaping child outcomes. Chapter 2 describes the 
sample and procedures used in the New Chance Observational Study. Chapter 3 presents data for 
the observational study sample regarding program participation, focusing especially on those 
program components with some content relevant to parenting. The chapter also provides qualita- 
tive data from retrospective interviews with field staff regarding the content of the parenting edu- 
cation component. Chapters 4 and 5 present findings on program impacts on the affective quality 
of mother-child interaction and literacy-related aspects of observed interaction, respectively. 
Chapter 6 completes the picture of program impacts on parenting measures by summarizing 
findings for the observational study sample on the interview-based measures of parenting. 
Chapter 7 looks across all of the available measures of parenting, comparing the magnitude of 
program impacts on each of the parenting measures; considering which program elements, if any, 
the program impacts on parenting variables were linked to; asking how closely the differing par- 
enting measures are correlated; and asking whether the observational measures add significantly 
to our ability to predict child outcomes beyond the measures of parenting derived from the inter- 
view context. Chapter 8 asks what role parenting behaviors — along with variables reflecting 
mothers’ human capital, mothers’ psychological well-being, and families’ current context — 
play in shaping child outcomes. Chapter 9 concludes Part I with a summary of findings and a 
discussion of their implications. 

Part II focuses on the measurement process. Chapter 10 explores the implications of con- 
ducting research on parenting, particularly observational research, using a survey research ap- 
proach, and considers how methodological research on survey measurement can inform such 
work. Chapter 1 1 discusses the process of translating protocols from developmental psychology 
into a format suitable for survey administration and summarizes the challenges that were en- 
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countered in fielding the study in this manner. Chapter 12 presents data on the quality of the sur- 
vey interviewers’ work and mothers’ subjective reactions to the observational session. Chapters 
13 and 14 conclude Part II by providing an evaluation of the success of our fielding strategy and 
extracting key recommendations for future research seeking to bridge the distance between inten- 
sive observational procedures and larger, more representative samples. 

Throughout the monograph we draw heavily on previous MDRC publications regarding 
the New Chance Program and Evaluation. We rely especially on New Chance: Implementing a 
Comprehensive Program for Disadvantaged Young Mothers and Their Children (Quint, Fink, 
and Rowser, 1991); New Chance: Interim Findings on a Comprehensive Program for Disad- 
vantaged Young Mothers and Their Children (Quint et al., 1994); Lives of Promise, Lives of 
Pain: Young Mothers After New Chance (Quint and Musick, 1994), and New Chance: Final Re- 
port on a Comprehensive Program for Young Mothers in Poverty land Their Children (Quint, 
Bos, and Polit, 1997). We also draw heavily on Two-Generation Programs for Families in Pov- 
erty: A New Intervention Strategy (Smith, 1995), which includes a chapter by Quint and Egeland 
on New Chance. 
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Chapter 2 

The Methodology of the New Chance Observational Study 
Donna R. Morrison , Carolyn A. Eldred, Martha J. Zaslow, and M. Robin Dion 



This chapter presents the methodology of the New Chance Observational 
Study. We begin by describing the sample for the observational study: how the 
290 families who participated in this study were selected, and what their charac- 
teristics were at baseline. Next we turn to an examination of program impacts on 
variables from the 18- and 42-month follow-up surveys. Whereas Chapter 1 pro- 
vided a summary of program impacts from these surveys for the full New Chance 
Evaluation sample, here we present impact findings specifically for the observa- 
tional study sample regarding maternal education, employment and economic 
self-sufficiency, fertility, and psychological well-being, as well as child outcomes. 

We then provide an overview of the procedures of the observational study, de- 
tailing the mother-child interaction tasks and the brief interview that took place at 
the time of the observational session, approximately 21 months after families en- 
rolled in the New Chance Evaluation. Finally, the chapter summarizes all of the 
parenting measures available for the observational study sample and gives an 
overview of the strategy of analysis used to address the specific questions of the 
New Chance Observational Study. 

Chapter 1 introduced the goals and specific questions of the New Chance Observational 
Study. In addition, it presented the context of the New Chance Observational Study by providing 
an overview of the New Chance Program and by summarizing the results available for the full 
evaluation of the program. We turn now to focus specifically on the methodology of the obser- 
vational study. We first describe the sample for the observational study, noting how it was se- 
lected, presenting sample characteristics, and summarizing impact findings from the 1 8- and 42- 
month follow-up surveys specifically for the embedded observational study sample. We then 
turn to descriptions of the procedures involved in the observational study and of the specific par- 
enting measures available for the observational study sample. We conclude with a summary of 
the analytic strategy used to address the questions of the New Chance Observational Study. 



I. The Sample 

A. The Identification of an Initial Pool of Families for the Observational Study 

The New Chance Observational Study was “embedded” within the larger New Chance 
Evaluation in order to provide detailed information about parenting behavior for a subset of New 
Chance sample members. For the observational study sample, data collected during the observa- 
tional session was combined with information collected during the 1 8- and 42-month follow-up 
surveys of sample members, as well as the participation data about experimental group members 
from the New Chance Management Information System. Maternal self-reports and interviewer 
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ratings from the follow-up surveys and from a brief interview carried out at the time of the ob- 
servational session complement data derived from the videotaped interactions of mothers and 
children during the observational session. 

Selection of the observational study sample sought to optimize the analytical value of the 
information to be collected within practical constraints. Resource constraints precluded including 
all 16 New Chance sites in such an intensive effort as the observational study (and one requiring 
costly equipment and props). Therefore, sample selection involved choosing both sites and re- 
spondents within sites. This was not entirely a sequential process, since characteristics of sample 
members in each site played a role in site selection. The potential pool of participants in the ob- 
servational study was narrowed on the basis of the age of the focal child, ethnicity, and timing of 
sample enrollment. We examined the potential contribution of each of the 16 New Chance sites 
to the observational study sample within these constraints. We hoped to limit the number of sites 
to perhaps five, but found it necessary to include seven sites in order to identify sufficient sample 
members to yield the target of 300 completed observational sessions. 

Although the focal children in the full New Chance Evaluation ranged in age from 18 
months to 8 years at the time of the 18-month follow-up, in the observational study we chose to 
focus on a narrower age range. As described in detail in Chapters 4 and 5 of this monograph, the 
mother-child interaction tasks used in the observational study had been used in Egeland and 
Snow’s research with preschool-age children. Focusing on children from the New Chance 
Evaluation who were approximately 30 to 60 months old at the time of the observational session 
allowed us to use the activities with which the research teams headed by Egeland and Snow had 
considerable experience and to include many of the focal children from the larger evaluation. 

We anticipated wanting to examine parenting behavior in light of ethnic differences 
within the observational study sample, because of evidence that group differences exist in pat- 
terns of mother-child interaction and in the relationship of such patterns to child outcomes (see 
Garcia Coll, 1990; Bradley et al., 1989). However, it turned out not to be possible to select sites 
in a way that would result in sufficient representation of the three predominant groups in the full 
evaluation sample (blacks, Hispanics, and whites). Both practical considerations (such as varia- 
tion in the sample enrollment schedule across sites) and the intent that this not be a study that 
examined parenting only among minority families led us to restrict the sample to black and white 
members of the larger New Chance sample. 

Another objective was to conduct each observational session as soon as possible after the 
18-month interview, but in no case more than four months later. With data collection for the ob- 
servational study to begin almost a year after the 18-month anniversary of the first monthly co- 
hort enrolled in the full evaluation, some respondents were already beyond the four-month 
maximum time period we wished to allow. In addition, sites varied with respect to the number of 
sample members ultimately enrolled and the speed with which they were enrolled over the two- 
year sample enrollment period. Thus, another criterion for evaluating sites for inclusion in the 
observational study was the size of the pool remaining after sample members enrolled during the 
first nine enrollment months were omitted. 

With the potential pool in each site narrowed based on age of the focal child, ethnicity, 
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and timing of sample enrollment, we chose seven sites' that would provide sufficient sample 
members to yield the target of 300 completed sessions. In each site, all of the black and white 
respondents with an 18-month anniversary date between November 1991 and January 1993 and a 
focal child who would be 30 to 60 months old at the time of the 18-month interview — or within 
four months of the interview — were included in a preliminary pool of families. 2 Experimentals 
constituted 63 percent of this preliminary pool of families and controls 37 percent. Although we 
had hoped to have a large enough pool to permit oversampling of controls (to provide more cases 
for analysis), we were not able to modify appreciably the two-to-one experimental-control ratio 
of the full evaluation sample. In all, 438 New Chance sample members were included in an ini- 
tial pool of families for inclusion in the observational study, based on the criteria above. 

B. Response Rates. Experimental Balance, and Representativeness of the Observa- 
tional Study Participants 

The initial pool of 438 families included 63 sample members who were ultimately con- 
sidered ineligible for participation in the study because they did not meet all of the study re- 
quirements, most notably completion of an 18-month interview with the focal child present. 3 A 
group of 375 families were then considered eligible for the study. Of this group, 297 actually 
participated in the observational study, for a response rate of 79 percent, lower than the 91 per- 
cent in the 18-month survey in the full evaluation. Twelve percent refused. 4 For the remaining 9 
percent of eligible respondents who did not participate, logistical problems in completing the 
field work were responsible; nearly two-thirds of these cases were simply not completed before 
the four-month interviewing “window” closed or the child “aged out” of the target age range; 5 
others could not be located or had moved from the area. 

Of the 297 completed cases, 290 resulted in codable videotapes. Technical difficulties oc- 
casionally rendered tapes unusable, as did some of the very early work of a few interviewers. 
Thus, the analyses described in this monograph are based on 77 percent of the eligible sample. 
As seen above, some, but not all, of this attrition was based on self-selection. 



'The Bronx, Detroit, Lexington, Harlem, Philadelphia, Pittsburgh, and Portland. 

2 The inclusion of children who had not yet “aged in” to eligibility or were about to “age out” required attention 
to maintain the integrity of the target age range. Nonetheless, 12 children who participated in the study fell outside 
the age range, resulting in an actual age range of 27 to 63 months. 

3 In 65 percent of these cases, no 18-month interview was completed at all, typically because the respondent 
could not be located, while in another 21 percent the randomly selected child was not in the home and there was no 
other child who could be substituted. In the remaining cases deemed ineligible, either the focal child from the 18- 
month interview was no longer in the home at the time of the observational session or selection of a substitute child 
for the 1 8-month interview or delays in completing that interview resulted in the child being outside the target age 
range. 

4 Obviously the demands and intrusiveness of the observational session were greater than for a traditional survey 
interview, no doubt leading some respondents to decline. In addition, concern about the potential intrusiveness of 
the work led us to counsel interviewers not to press as hard to “convert” those who initially refused as they might in 
more traditional survey work. 

5 These cases no doubt included an undetermined number of “soft” refusals, that is, those who did not actually 
refuse but broke appointments, could not find a convenient time for the session, and so on. 
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The group of 290 cases on which the analyses are based includes 184 experimentals and 
106 controls, the same ratio of 63 to 37 percent as in the initial pool of 438 families. Additional 
analyses were undertaken to assess the comparability of the experimental and control group 
members who participated in the observational study. 

Mothers in the full evaluation sample had been randomly assigned to either the experi- 
mental or control group; analyses carried out with the full sample confirm that these groups did 
not differ significantly in terms of baseline characteristics (see Quint, Polit, Bos and Cave, 1994, 
Appendix A). Parallel analyses were carried out for the observational study sample. Here we fo- 
cus on the 290 cases actually included in analyses for the New Chance Observational Study, and 
thus do not include the 7 cases with unusable videotapes. 

Regression models were estimated in which the dependent variable was a dummy indi- 
cating experimental status (that is, membership in the experimental group versus the control 
group) and the independent variables were characteristics of the children, mothers, and families 
at baseline. The resulting F-statistic was not statistically significant, indicating that any differ- 
ence in baseline characteristics between experimentals and controls in the observational study 
sample can be attributed to chance (see Appendix Table A.l). Thus, any differences observed 
between the groups on the measures in this study cannot be attributed to prior group differences. 

We noted that self-selection was one factor in the proportion of eligible families who 
participated in the observational study. In addition, the denominator of cases ultimately deter- 
mined to be “eligible” may itself have been affected by self-selection (such as failure to partici- 
pate in the 18-month survey) and characteristics holding potential substantive interest (such as 
the focal child not living with the mother). Thus, we undertook additional analyses to assess the 
representativeness of the observational study participants. Here, rather than focusing on the 290 
cases included in our data analyses, our sample is the larger group of 297 who actually com- 
pleted the observational study procedures. We include here the 7 cases that completed the proce- 
dures of the observational study but whose videotapes were unusable because the underlying 
question is whether those families who actually participated in the study differed from those in 
the initial pool in the seven sites who did not participate. 

Accordingly, we compared the baseline characteristics of these 297 families with those of 
other families in the seven study sites included in the initial pool of families who did not ulti- 
mately participate (n = 141). Families who participated in the observational study differed sig- 
nificantly from those in the initial pool of families who did not participate with regard to welfare 
status at baseline (see Appendix Table B.l). Mothers who completed the observational study 
procedures were less likely to be receiving welfare at baseline. No other markers of disadvantage 
at baseline (for example, mothers’ education, employment, receipt of welfare as a child) differed 
across these two groups. These findings point to little difference between those who participated 
in the New Chance Observational Study and those in the initial pool of families who did not par- 
ticipate beyond what might be expected by chance. 6 



6 These analyses were replicated contrasting the 290 families whose data were actually included in analyses for 
the New Chance Observational Study with 148 families for whom we do not have data (here including the 7 fami- 

(continued) 
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